Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator

Hekimoglu M. B., Alioğlu A., Filiz U., Ulusoy I., Schmidt K. V.

33rd IEEE Conference on Signal Processing and Communications Applications, SIU 2025, İstanbul, Türkiye, 25 - 28 Haziran 2025, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/siu66497.2025.11112292
  • Basıldığı Şehir: İstanbul
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: Building Energy Management, Data-Driven Models, Reinforcement Learning, Training Acceleration
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

Reinforcement learning (RL) has shown great potential in optimizing the operation of HVAC systems, improving energy efficiency, and enhancing user comfort. However, the slow input/output operations associated with simulation tools like EnergyPlus significantly hinder the training process. This paper proposes a novel approach to accelerate RL training by using a data-driven LSTM model to replicate the behavior of a building energy simulator. By training the LSTM model on a set of observations and actions, the model learns to approximate the simulator's dynamics, providing a faster and more efficient training environment for RL agents. We demonstrate that using the LSTM-based surrogate simulator leads to substantial reductions in computational time while maintaining the accuracy of the system's behavior.