Improving reinforcement learning by using sequence trees

Creative Commons License

Girgin S., POLAT F., Alhajj R.

MACHINE LEARNING, cilt.81, sa.3, ss.283-331, 2010 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 81 Sayı: 3
  • Basım Tarihi: 2010
  • Doi Numarası: 10.1007/s10994-010-5182-y
  • Dergi Adı: MACHINE LEARNING
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Sayfa Sayıları: ss.283-331
  • Anahtar Kelimeler: Reinforcement learning, Options, Conditionally terminating sequences, Temporal abstractions, Semi-Markov decision processes, SOCCER, ABSTRACTION
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

This paper proposes a novel approach to discover options in the form of stochastic conditionally terminating sequences; it shows how such sequences can be integrated into the reinforcement learning framework to improve the learning performance. The method utilizes stored histories of possible optimal policies and constructs a specialized tree structure during the learning process. The constructed tree facilitates the process of identifying frequently used action sequences together with states that are visited during the execution of such sequences. The tree is constantly updated and used to implicitly run corresponding options. The effectiveness of the method is demonstrated empirically by conducting extensive experiments on various domains with different properties.