Abstract:Integrating unmanned aerial vehicles (UAVs) into existing cellular networks encounters lots of challenges, among which one of the most striking concerns is how to achieve harmonious coexistence of aerial transceivers, inter alia, UAVs, and terrestrial user equipments (UEs). In this paper, a cellular-connected UAV network is focused, where multiple UAVs receive messages from base stations (BSs) in the down-link, while BSs are serving ground UEs in their cells. For effectively managing inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first $p$-tier based RB coordination criterion is proposed and adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design minimizing UAV's ergodic outage duration (EOD) is investigated. To cope with conventional optimization techniques' inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is initiated, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in discrete action domain and beamforming design in continuous action regime, respectively. The hybrid D3QN-TD3 solution is trained via interacting with the considered outer and inner environments in an online centralized manner so that it can then help achieve the suboptimal EOD minimization performance during its offline decentralized exploitation phase. Simulation results have illustrated the effectiveness of the proposed hybrid D3QN-TD3 algorithm, compared to several representative baselines.

The Transmission Scheme for UAV MIMO System Based on Reinforcement Learning

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Three-Dimensional Trajectory and Resource Allocation Optimization in Multi-Unmanned Aerial Vehicle Multicast System: A Multi-Agent Reinforcement Learning Method

Learning-Enabled Radar-Assisted Predictive Beamforming for UAV-Aided Networks

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

CoMP and RIS-assisted Multicast Transmission in a Multi-UAV Communication System

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

UAV-Assisted Enhanced Coverage and Capacity in Dynamic MU-mMIMO IoT Systems: A Deep Reinforcement Learning Approach

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Multi-Agent Learning Based Packet Routing in Multi-Hop UAV Relay Network

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Resource Scheduling Based on Deep Reinforcement Learning in UAV Assisted Emergency Communication Networks

Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach

Deep Reinforcement Learning for Beam Management in UAV Relay mmWave Networks

Millimeter-Wave Networking in the Sky: A Machine Learning and Mean Field Game Approach for Joint Beamforming and Beam-Steering

Multi-Agent Deep Reinforcement Learning for Secure UAV Communications