State Similarity Based Approach for Improving Performance in RL

Girgin S., POLAT F., Alhajj R.

20th International Joint Conference on Artificial Intelligence, Hyderabad, Pakistan, 6 - 12 January 2007, pp.817-822 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • City: Hyderabad
  • Country: Pakistan
  • Page Numbers: pp.817-822
  • Middle East Technical University Affiliated: Yes


This paper employs state similarity to improve reinforcement learning performance. This is achieved by first identifying states with similar sub-policies. Then, a tree is constructed to be used for locating common action sequences of states as derived from possible optimal policies. Such sequences are utilized for defining a similarity function between states, which is essential for reflecting updates on the action-value function of a state onto all similar states. As a result, the experience acquired during learning can be applied to a broader context. Effectiveness of the method is demonstrated empirically.