Routing-Aware RL for Mobile Relay Navigation in Disconnected Ad Hoc Networks
21st International Conference on Network and Service Management, CNSM 2025, Bologna, Hindistan, 27 - 31 Ekim 2025, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.23919/cnsm67658.2025.11297454
- Basıldığı Şehir: Bologna
- Basıldığı Ülke: Hindistan
- Anahtar Kelimeler: ad hoc networks, curriculum learning, distributional RL, domain randomization, mobile relays, OMNeT++, PPO, QRDQN, reinforcement learning, routingaware observation
- Orta Doğu Teknik Üniversitesi Adresli: Evet
Özet
Mobile relays can restore end-to-end connectivity in disconnected ad hoc networks, yet classical placement methods rely on global geometry and channel maps that are rarely available in practice. We study routing-aware reinforcement learning (RL) for a single controlled host that moves on a 2D plane to bridge disconnected clusters and enable multi-hop delivery between a source and destination. Built on OMNeT++/INET, our environment exposes only local, router-observable signals and augments them with a lightweight host-discovery memory. We propose a composite reward that couples end-to-end delivery with topology-shaping incentives. We benchmark PPO and QRDQN under domain randomization and a scenario curriculum that rotates layouts every K episodes. Both methods learn to discover hosts, position between clusters, and sustain high forwarding efficiency on unseen topologies; QRDQN converges faster under sparser rewards while PPO yields smoother final policies.