A novel deep reinforcement learning algorithm for online antenna tuning

Balevi E., Andrews J. G.

2019 IEEE Global Communications Conference, GLOBECOM 2019, Hawaii, Amerika Birleşik Devletleri, 9 - 13 Aralık 2019 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/globecom38437.2019.9013308
  • Basıldığı Şehir: Hawaii
  • Basıldığı Ülke: Amerika Birleşik Devletleri
  • Anahtar Kelimeler: Deep reinforcement learning, HetNets, Online antenna tuning, Q-learning
  • Orta Doğu Teknik Üniversitesi Adresli: Hayır

Özet

© 2019 IEEE.The interactions between the cells, most notably due to their coupled interference and the large number of users, render the optimization of antenna parameters prohibitively complex. To cope with this problem, we propose a novel practical deep learning (DL) based reinforcement learning (RL) algorithm to jointly optimize antenna tilt angle and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). In the proposed algorithm, DL is used to extract the features by learning the locations of users, and mean field RL is used to learn the average interference values for different antenna settings. Our results illustrate that the proposed deep RL algorithm can approach the optimum weighted sum rate with hundreds of online trials, as opposed to millions of trials for standard Q-learning, assuming relatively low environmental dynamics. Furthermore, the proposed algorithm is compact and implementable, and empirically appears to provide a performance guarantee regardless of the amount of environmental dynamics.