A reinforcement learning based algorithm for personalization of digital, just-in-time, adaptive interventions


Gönül S., Namlı T., Coşar A., TOROSLU İ. H.

Artificial Intelligence in Medicine, cilt.115, 2021 (SCI-Expanded) identifier identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 115
  • Basım Tarihi: 2021
  • Doi Numarası: 10.1016/j.artmed.2021.102062
  • Dergi Adı: Artificial Intelligence in Medicine
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Aerospace Database, Applied Science & Technology Source, Biotechnology Research Abstracts, CINAHL, Communication Abstracts, Compendex, Computer & Applied Sciences, EMBASE, INSPEC, Library, Information Science & Technology Abstracts (LISTA), MEDLINE, Metadex, Psycinfo, Civil Engineering Abstracts
  • Anahtar Kelimeler: Reinforcement learning, Intervention delivery optimization, Just-in-time adaptive interventions, Personalized health interventions, Personalized interventions, BEHAVIOR, MODELS
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

© 2021 Elsevier B.V.Suboptimal health related behaviors and habits; and resulting chronic diseases are responsible for majority of deaths globally. Studies show that providing personalized support to patients yield improved results by preventing and/or timely treatment of these problems. Digital, just-in-time and adaptive interventions are mobile phone-based notifications that are being utilized to support people wherever and whenever necessary in coping with their health problems. In this research, we propose a reinforcement learning-based mechanism to personalize interventions in terms of timing, frequency and preferred type(s). We simultaneously employ two reinforcement learning models, namely intervention-selection and opportune-moment-identification; capturing and exploiting changes in people's long-term and momentary contexts respectively. While the intervention-selection model adapts the intervention delivery with respect to type and frequency, the opportune-moment-identification model tries to find the most opportune moments to deliver interventions throughout a day. We propose two accelerator techniques over the standard reinforcement learning algorithms to boost learning performance. First, we propose a customized version of eligibility traces for rewarding past actions throughout an agent's trajectory. Second, we utilize the transfer learning method to reuse knowledge across multiple learning environments. We validate the proposed approach in a simulated experiment where we simulate four personas differing in their daily activities, preferences on specific intervention types and attitudes towards the targeted behavior. Our experiments show that the proposed approach yields better results compared to the standard reinforcement learning algorithms and successfully capture the simulated variations associated with the personas.