Positive impact of state similarity on reinforcement learning performance


Girgin S., Polat F. , Alhaj R.

IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS, cilt.37, ss.1256-1270, 2007 (SCI İndekslerine Giren Dergi) identifier identifier identifier

  • Cilt numarası: 37 Konu: 5
  • Basım Tarihi: 2007
  • Doi Numarası: 10.1109/tsmcb.2007.899419
  • Dergi Adı: IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS
  • Sayfa Sayıları: ss.1256-1270

Özet

In this paper, we propose a novel approach to identify states with similar subpolicies and show how they can be integrated into the reinforcement learning framework to improve learning performance. The method utilizes a specialized tree structure to identify common action sequences of states, which are derived from possible optimal policies, and defines a similarity function between two states based on the number of such sequences. Using this similarity function, updates on the action-value function of a state are reflected onto all similar states. This allows experience that is acquired during learning to be applied to a broader context. The effectiveness of the method is demonstrated empirically.