Abstract:Integrating unmanned aerial vehicles (UAVs) into existing cellular networks encounters lots of challenges, among which one of the most striking concerns is how to achieve harmonious coexistence of aerial transceivers, inter alia, UAVs, and terrestrial user equipments (UEs). In this paper, a cellular-connected UAV network is focused, where multiple UAVs receive messages from base stations (BSs) in the down-link, while BSs are serving ground UEs in their cells. For effectively managing inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first $p$-tier based RB coordination criterion is proposed and adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design minimizing UAV's ergodic outage duration (EOD) is investigated. To cope with conventional optimization techniques' inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is initiated, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in discrete action domain and beamforming design in continuous action regime, respectively. The hybrid D3QN-TD3 solution is trained via interacting with the considered outer and inner environments in an online centralized manner so that it can then help achieve the suboptimal EOD minimization performance during its offline decentralized exploitation phase. Simulation results have illustrated the effectiveness of the proposed hybrid D3QN-TD3 algorithm, compared to several representative baselines.

Resource Scheduling Based on Deep Reinforcement Learning in UAV Assisted Emergency Communication Networks

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

UAV-Assisted 5G/6G Networks: Joint Scheduling and Resource Allocation Based on Asynchronous Reinforcement Learning

Multi-Agent DRL for Air-to-Ground Communication Planning in UAV-Enabled IoT Networks

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Priority-aware path planning and user scheduling for UAV-mounted MEC networks: A deep reinforcement learning approach

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

The Transmission Scheme for UAV MIMO System Based on Reinforcement Learning

Deep Reinforcement Learning Enabled Multi-UAV Scheduling for Disaster Data Collection With Time-Varying Value

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Dynamic resource allocation in IRS-assisted UAV wideband cognitive radio networks: A DDQN-TD3 approach

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

UAV-Assisted Wireless Cooperative Communication and Coded Caching: A Multiagent Two-Timescale DRL Approach

Autonomous Navigation and Configuration of Integrated Access Backhauling for UAV Base Station Using Reinforcement Learning