Context-aware Markov decision processes

---

Tezin Türü: Yüksek Lisans

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü, Türkiye

Tezin Onay Tarihi: 2014

Öğrenci: ÖMER EKMEKCİ

Danışman: FARUK POLAT

Özet:

In the 1990s, when artificial intelligence (AI) research become an important area again, researchers quitted trying to solve the ultimate problem, generating autonomous agents with "human-level intelligence". Currently, significantly important part of the research is highly focused on developing autonomous agents particularly dedicated to solve problems only in a chosen domain. Even though models and algorithms provided in reinforcement learning (RL), such as Markov Decision Processes (MDP), are successful at efficiently determining optimal or near optimal policies for these problems, however, these tools, originated from operations research and not customized particularly for AI, ignore using the available information in a given problem which indeed makes them far away from being a suitable model for AI. This leads unstructured representation of that problem which makes these tools significantly less efficient at determining a useful policy as the state space of a task grows, which is the case for more realistic problems.A milestone will be achieved in AI if new state machines are invented that use the information in a given task enabling generate optimal or near-optimal policies up to more realistic tasks having large number of states.If this is successfully achieved, the research will be one step closer to fulfill the ultimate aim of AI. Based on this motivation, this thesis presents a new state machine, based on MDP, for representing and solving more realistic AI problems which is entitled "Context-Aware Markov Decision Process (CA-MDP)". For that matter, CA-MDP, in comparison to MDP, introduces information based on causal relationship of actions and states therefore enabling compact represention of the tasks and computation of an optimal policy much more efficiently, even for problems having very large number of states that MDP fails to solve efficiently which will make it an important step in integrating the information available to both representation and solution of an AI problem. After a theoretical introduction of the new state machine, an analysis is carried out. Finally, the effectiveness of the model is demonstrated experimentally, concluding with a comparison to existing models.