Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator

Hekimoglu M. B., Alioğlu A., Filiz U., Ulusoy I., Schmidt K. V.

33rd IEEE Conference on Signal Processing and Communications Applications, SIU 2025, İstanbul, Turkey, 25 - 28 June 2025, (Full Text) identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/siu66497.2025.11112292
  • City: İstanbul
  • Country: Turkey
  • Keywords: Building Energy Management, Data-Driven Models, Reinforcement Learning, Training Acceleration
  • Middle East Technical University Affiliated: Yes

Abstract

Reinforcement learning (RL) has shown great potential in optimizing the operation of HVAC systems, improving energy efficiency, and enhancing user comfort. However, the slow input/output operations associated with simulation tools like EnergyPlus significantly hinder the training process. This paper proposes a novel approach to accelerate RL training by using a data-driven LSTM model to replicate the behavior of a building energy simulator. By training the LSTM model on a set of observations and actions, the model learns to approximate the simulator's dynamics, providing a faster and more efficient training environment for RL agents. We demonstrate that using the LSTM-based surrogate simulator leads to substantial reductions in computational time while maintaining the accuracy of the system's behavior.