Novel Q-learning-based neighborhood discovery algorithm for directed antennas
Array, vol.30, 2026 (ESCI, Scopus)
- Publication Type: Article / Article
- Volume: 30
- Publication Date: 2026
- Doi Number: 10.1016/j.array.2026.100827
- Journal Name: Array
- Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus, Compendex, Directory of Open Access Journals
- Keywords: Directed antenna, Neighborhood discovery, Q-learning, Vizier's Theorem
- Middle East Technical University Affiliated: Yes
Abstract
Directed antennas offer significant energy savings, yet the neighbor-discovery phase requires nontrivial algorithms. Using graph-theoretic analysis, we establish theoretical lower and upper bounds on the time required to reach minimally connected antenna topologies. We develop a Q-learning-based (QL) neighbor-discovery algorithm and introduce two enhancements. First, we propose an adaptive mechanism that dynamically updates the exploration rate ɛ based on changes in Q-tables, enabling a principled balance between exploration and exploitation. Second, we introduce a temporal sector-pruning heuristic that penalizes and prunes unfruitful sector pairs. Experiments are performed in a custom simulator, measuring the time required to reach minimal connectivity. We compare the proposed methods against random walk and baseline QL across regular, dense (King Grid), and irregular, sparse (Random) topologies. Results show that the synergistic integration of pruning and adaptive QL reduces discovery time by about one-third relative to QL on regular and dense graphs. On sparse graphs, baseline QL fails to complete within the threshold time, while the synergistic scheme achieves at least a tenfold speed-up. The synergistic method further outperforms pruning-only QL by 30%–86% and adaptive-only QL by 7%–16% in discovery time.