GENERATING EFFECTIVE INITIATION SETS FOR SUBGOAL-DRIVEN OPTIONS

DEMİR A., Cilden E., POLAT F.

ADVANCES IN COMPLEX SYSTEMS, vol.22, no.2, 2019 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 22 Issue: 2
  • Publication Date: 2019
  • Doi Number: 10.1142/s0219525919500012
  • Journal Name: ADVANCES IN COMPLEX SYSTEMS
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Keywords: Reinforcement learning, options framework, option initiation set, subgoal discovery, Markov decision process, REINFORCEMENT, CUT
  • Middle East Technical University Affiliated: Yes

Abstract

Options framework is one of the prominent models serving as a basis to improve learning speed by means of temporal abstractions. An option is mainly composed of three elements: initiation set, option's local policy and termination condition. Although various attempts exist that focus on how to derive high-quality termination conditions for a given problem, the impact of initiation set generation is relatively unexplored. In this work, we propose an effective goal-oriented heuristic method to derive useful initiation set elements via an analysis of the recent history of events. Effectiveness of the method is experimented on various benchmark problems, and the results are discussed.