A novel deep reinforcement learning algorithm for online antenna tuning

Balevi E., Andrews J. G.

2019 IEEE Global Communications Conference, GLOBECOM 2019, Hawaii, United States Of America, 9 - 13 December 2019 identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/globecom38437.2019.9013308
  • City: Hawaii
  • Country: United States Of America
  • Keywords: Deep reinforcement learning, HetNets, Online antenna tuning, Q-learning
  • Middle East Technical University Affiliated: No


© 2019 IEEE.The interactions between the cells, most notably due to their coupled interference and the large number of users, render the optimization of antenna parameters prohibitively complex. To cope with this problem, we propose a novel practical deep learning (DL) based reinforcement learning (RL) algorithm to jointly optimize antenna tilt angle and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). In the proposed algorithm, DL is used to extract the features by learning the locations of users, and mean field RL is used to learn the average interference values for different antenna settings. Our results illustrate that the proposed deep RL algorithm can approach the optimum weighted sum rate with hundreds of online trials, as opposed to millions of trials for standard Q-learning, assuming relatively low environmental dynamics. Furthermore, the proposed algorithm is compact and implementable, and empirically appears to provide a performance guarantee regardless of the amount of environmental dynamics.