A Concept Filtering Approach for Diverse Density to Discover Subgoals in Reinforcement Learning

DEMİR A., Cilden E., POLAT F.

IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI), Massachusetts, United States Of America, 6 - 08 November 2017, pp.1-5 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/ictai.2017.00012
  • City: Massachusetts
  • Country: United States Of America
  • Page Numbers: pp.1-5
  • Keywords: reinforcement learning, subgoal discovery, diverse density, NETWORKS


In the reinforcement learning context, subgoal discovery methods aim to find bottlenecks in problem state space so that the problem can naturally be decomposed into smaller subproblems. In this paper, we propose a concept filtering method that extends an existing subgoal discovery method, namely diverse density, to be used for both fully and partially observable RL problems. The proposed method is successful in discovering useful subgoals with the help of multiple instance learning. Compared to the original algorithm, the resulting approach runs significantly faster without sacrificing the solution quality. Moreover, it can effectively be employed to find observational bottlenecks of problems with perceptually aliased states.