Abstract:Integrating unmanned aerial vehicles (UAVs) into existing cellular networks encounters lots of challenges, among which one of the most striking concerns is how to achieve harmonious coexistence of aerial transceivers, inter alia, UAVs, and terrestrial user equipments (UEs). In this paper, a cellular-connected UAV network is focused, where multiple UAVs receive messages from base stations (BSs) in the down-link, while BSs are serving ground UEs in their cells. For effectively managing inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first $p$-tier based RB coordination criterion is proposed and adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design minimizing UAV's ergodic outage duration (EOD) is investigated. To cope with conventional optimization techniques' inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is initiated, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in discrete action domain and beamforming design in continuous action regime, respectively. The hybrid D3QN-TD3 solution is trained via interacting with the considered outer and inner environments in an online centralized manner so that it can then help achieve the suboptimal EOD minimization performance during its offline decentralized exploitation phase. Simulation results have illustrated the effectiveness of the proposed hybrid D3QN-TD3 algorithm, compared to several representative baselines.

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Distributed reinforcement learning based framework for energy-efficient UAV relay against jamming

Meta Reinforcement Learning for Resource Allocation in Aerial Active-RIS-assisted Networks with Rate-Splitting Multiple Access

Covert UAV Data Transmission Via Semantic Communication: A DRL-Driven Joint Position and Power Optimization Method

Energy Harvesting Reconfigurable Intelligent Surface for UAV Based on Robust Deep Reinforcement Learning

Joint Trajectory and Passive Beamforming Design for Secure UAV Networks with RIS

Reinforcement Learning Based Energy Efficient Robot Relay for Unmanned Aerial Vehicles Against Smart Jamming

RIS-Assisted UAV-Enabled Wireless Powered Communications: System Modeling and Optimization

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

UAV-Mounted RIS-Aided Mobile Edge Computing System: A DDQN-Based Optimization Approach

UAV-Assisted Enhanced Coverage and Capacity in Dynamic MU-mMIMO IoT Systems: A Deep Reinforcement Learning Approach

Energy Efficient Design in IRS-Assisted UAV Data Collection System under Malicious Jamming

A deep reinforcement learning framework and its implementation for UAV-aided covert communication

Energy-Efficient Symbiotic UAV-Enabled MEC Networks via RIS: Joint Trajectory and Phase-Shift Control Optimization

RIS-assisted UAV Communications for IoT with Wireless Power Transfer Using Deep Reinforcement Learning