Subgoal identification with multiple instance learning methods in landmark Partially Observable Markov Decision Process problems
Knowledge-Based Systems, cilt.346, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 346
- Basım Tarihi: 2026
- Doi Numarası: 10.1016/j.knosys.2026.116178
- Dergi Adı: Knowledge-Based Systems
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC, Library, Information Science & Technology Abstracts (LISTA)
- Anahtar Kelimeler: Multiple instance learning, Partially Observable Markov Decision Process (POMDP), Reinforcement learning, Subgoal identification
- Orta Doğu Teknik Üniversitesi Adresli: Evet
Özet
In the literature, subgoal identification methods have been shown to improve the learning performance of reinforcement learning (RL) agents for Markov Decision Process (MDP) problems. Most methods rely on the distinctiveness of states; therefore, they cannot be used in Partially Observable Markov Decision Process (POMDP) problems where state information is ambiguous. However, only a few multiple instance learning (MIL)-based methods (baseline methods) have been shown to identify subgoals in landmark POMDP problems where some states emit unique observations. These techniques rely on early MIL algorithms and require distance information between data instances, which can be misleading in POMDP problems because of ambiguity. The MIL domain has matured with many studies, and a large set of approaches can be considered for identifying subgoals in POMDP problems. To this end, we evaluate a wide range of other MIL-based methods that do not require distance information and make different assumptions about dependency relationships in datasets on several landmark POMDP problems. In addition to using existing methods, we propose new MIL methods and an architectural layer that leverage the dependencies that emerge in an RL setting. Furthermore, we experiment with different subgoal selection strategies based on the information the methods provide and how it can be summarized, considering the RL setting. Our results show that almost all methods employed and proposed surpass the baseline methods in average subgoal identification accuracy. As a related work, we also demonstrate that identified subgoals can enhance learning efficiency. The implementation and datasets are available at https://github.com/SaimSUNEL/SubgoalIdentificationwithMILMethodsinLandmarkPomdps.