Abstract:Integrating unmanned aerial vehicles (UAVs) into existing cellular networks encounters lots of challenges, among which one of the most striking concerns is how to achieve harmonious coexistence of aerial transceivers, inter alia, UAVs, and terrestrial user equipments (UEs). In this paper, a cellular-connected UAV network is focused, where multiple UAVs receive messages from base stations (BSs) in the down-link, while BSs are serving ground UEs in their cells. For effectively managing inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first $p$-tier based RB coordination criterion is proposed and adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design minimizing UAV's ergodic outage duration (EOD) is investigated. To cope with conventional optimization techniques' inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is initiated, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in discrete action domain and beamforming design in continuous action regime, respectively. The hybrid D3QN-TD3 solution is trained via interacting with the considered outer and inner environments in an online centralized manner so that it can then help achieve the suboptimal EOD minimization performance during its offline decentralized exploitation phase. Simulation results have illustrated the effectiveness of the proposed hybrid D3QN-TD3 algorithm, compared to several representative baselines.

Enabling Urban MmWave Communications with UAV-Carried IRS Via Deep Reinforcement Learning

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

Joint Beamforming and Trajectory Optimization for UAV-Assisted Double IRS Secure Transmission System: A Deep Reinforcement Learning Approach.

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Advancing reliability and efficiency of urban communication: Unmanned aerial vehicles, intelligent reflection surfaces, and deep learning techniques

Reinforcement-learning-based Trajectory Design and Phase-shift Control in UAV-mounted-RIS Communications

Deep reinforcement learning for IRS-assisted UAV covert communications

RIS-Assisted Robust Beamforming for UAV Anti-Jamming and Eavesdropping Communications: A Deep Reinforcement Learning Approach

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

A Hierarchical Deep Reinforcement Learning Approach for Throughput Maximization in Reconfigurable Intelligent Surface-Aided Unmanned Aerial Vehicle–Integrated Sensing and Communication Network

Deep Reinforcement Learning-Driven Reconfigurable Intelligent Surface-Assisted Radio Surveillance with a Fixed-Wing UAV.

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Optimal Design for Trajectory and Phase-Shift in UAV-Mounted-RIS Communications with Reinforcement Learning

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Dynamic Resource Allocation for MmWave UAV Communications: A Deep Reinforcement Learning Approach

Multi‐agent Reinforcement Learning Based Transmission Scheme for IRS‐assisted Multi‐uav Systems

Time-Effective UAV-IRS-Collaborative Data Harvesting: A Robust Deep Reinforcement Learning Approach

Distributional Reinforcement Learning for Mmwave Communications with Intelligent Reflectors on a UAV.

IRS Empowered UAV Wireless Communication with Resource Allocation, Reflecting Design and Trajectory Optimization