Compact Frequency Memory for Reinforcement Learning with Hidden States.

Polat F., Aydin H., Cilden E.

PRIMA 2019: Principles and Practice of Multi-Agent Systems - 22nd International Conference, Taranto, Italy, 28 - 31 October 2019, pp.425-433 identifier identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1007/978-3-030-33792-6_26
  • City: Taranto
  • Country: Italy
  • Page Numbers: pp.425-433
  • Keywords: Reinforcement Learning, Memory-based learning, Compact Frequency Memory
  • Middle East Technical University Affiliated: Yes


Memory-based reinforcement learning approaches keep track of past experiences of the agent in environments with hidden states. This may require extensive use of memory that limits the practice of these methods in a real-life problem. The motivation behind this study is the observation that less frequent transitions provide more reliable information about the current state of the agent in ambiguous environments. In this work, a selective memory approach based on the frequencies of transitions is proposed to avoid keeping the transitions which are unrelated to the agent’s current state. Experiments show that the usage of a compact and selective memory may improve and speed up the learning process.