Abstract:Modern cellular networks are multi-cell and use universal frequency reuse to maximize spectral efficiency. This results in high inter-cell interference. This problem is growing as cellular networks become three-dimensional with the adoption of unmanned aerial vehicles (UAVs). This is because the strength and number of interference links rapidly increase due to the line-of-sight channels in UAV communications. Existing interference management solutions need each transmitter to know the channel information of interfering signals, rendering them impractical due to excessive signaling overhead. In this paper, we propose leveraging deep reinforcement learning for interference management to tackle this shortcoming. In particular, we show that interference can still be effectively mitigated even without knowing its channel information. We then discuss novel approaches to scale the algorithms with linear/sublinear complexity and decentralize them using multi-agent reinforcement learning. By harnessing interference, the proposed solutions enable the continued growth of civilian UAVs.

What problem does this paper attempt to address?

The paper primarily aims to address the severe interference issues caused by Unmanned Aerial Vehicles (UAVs) in three-dimensional (3D) cellular networks. Specifically: 1. **Research Background**: With the development of UAV technology, the application of UAVs in cellular networks is becoming increasingly widespread, such as for package delivery, sensing tasks, etc. However, the altitude of UAVs leads to communication links that are predominantly line-of-sight, thereby increasing the intensity of air-to-ground and ground-to-air interference. 2. **Existing Challenges**: Current interference management methods rely on precise Channel State Information (CSI), which is difficult to achieve in practical deployments due to the significant signaling overhead and the requirement for high synchronization among nodes. Additionally, traditional techniques like interference alignment and Coordinated Multi-Point (CoMP), although theoretically effective, become impractical in practice due to the difficulty in obtaining CSI. 3. **Proposed Solution**: The paper suggests using Deep Reinforcement Learning (DRL) to overcome these limitations. DRL can make decisions in complex dynamic environments and can learn effective interference management strategies without requiring strict CSI. By defining appropriate agents, environments, states, actions, and reward mechanisms, DRL can not only achieve the functionality of traditional algorithms but also surpass them by reducing interference while improving spectral efficiency. 4. **Specific Implementation**: The article discusses in detail how to design DRL algorithms to handle discrete or continuous action space problems and proposes the Wolpertinger architecture to address the challenges posed by large-scale action spaces. Additionally, it discusses methods for distributed training within a multi-agent learning framework to reduce communication overhead and improve algorithm robustness. In summary, the paper aims to utilize deep reinforcement learning technology to solve the severe interference problems faced by UAVs in 3D cellular networks by developing new algorithms to enhance system performance while maintaining low computational complexity and signaling overhead.

Deep Reinforcement Learning for Interference Management in UAV-based 3D Networks: Potentials and Challenges

Cellular-Connected UAVs over 5G: Deep Reinforcement Learning for Interference Management

Downlink Transmit Power Control in Ultra-Dense UAV Network Based on Mean Field Game and Deep Reinforcement Learning

Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Mean-Field Multi-Agent Reinforcement Learning for UAV Assisted Secure Data Dissemination.

AI-Aided Downlink Interference Control in Dense Interference-Aware Drone Small Cells Networks

Mean Field Deep Reinforcement Learning for Fair and Efficient UAV Control

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

Trajectory Design and Resource Allocation for Multi-UAV Networks: Deep Reinforcement Learning Approaches

Deep Reinforcement Learning for Beam Management in UAV Relay mmWave Networks

Enabling Urban MmWave Communications with UAV-Carried IRS Via Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Intrusion Detection in Aerial Computing Networks.

Joint Optimization of Multi-UAV Deployment and User Association Via Deep Reinforcement Learning for Long-Term Communication Coverage

Decentralized Trajectory and Power Control Based on Multi-Agent Deep Reinforcement Learning in UAV Networks

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Multi-UAV Dynamic Wireless Networking with Deep Reinforcement Learning

Connectivity-Aware 3D UAV Path Design With Deep Reinforcement Learning

UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning

Deep Reinforcement Learning for UAV-Assisted Covert Data Dissemination.