Effectiveness of considering state similarity for reinforcement learning


Girgin S., Polat F. , Alhajj R.

INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING - IDEAL 2006, PROCEEDINGS, vol.4224, pp.163-171, 2006 (Journal Indexed in SCI) identifier

  • Publication Type: Article / Article
  • Volume: 4224
  • Publication Date: 2006
  • Title of Journal : INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING - IDEAL 2006, PROCEEDINGS
  • Page Numbers: pp.163-171

Abstract

This paper presents a novel approach that locates states with similar sub-policies, and incorporates them into the reinforcement learning framework for better learning performance. This is achieved by identifying common action sequences of states, which are derived from possible optimal policies and reflected into a tree structure. Based on the number of such sequences, we define a similarity function between two states, which helps to reflect updates on the action-value function of a state to all similar states. This way, experience acquired during learning can be applied to a broader context. The effectiveness of the method is demonstrated empirically.