Abstract:In this paper, we present a novel distributed UAVs beam reforming approach to dynamically form and reform a space-selective beam path in addressing the coexistence with satellite and terrestrial communications. Despite the unique advantage to support wider coverage in UAV-enabled cellular communications, the challenges reside in the array responses' sensitivity to random rotational motion and the hovering nature of the UAVs. A model-free reinforcement learning (RL) based unified UAV beam selection and tracking approach is presented to effectively realize the dynamic distributed and collaborative beamforming. The combined impact of the UAVs' hovering and rotational motions is considered while addressing the impairment due to the interference from the orbiting satellites and neighboring networks. The main objectives of this work are two-fold: first, to acquire the channel awareness to uncover its impairments; second, to overcome the beam distortion to meet the quality of service (QoS) requirements. To overcome the impact of the interference and to maximize the beamforming gain, we define and apply a new optimal UAV selection algorithm based on the brute force criteria. Results demonstrate that the detrimental effects of the channel fading and the interference from the orbiting satellites and neighboring networks can be overcome using the proposed approach. Subsequently, an RL algorithm based on Deep Q-Network (DQN) is developed for real-time beam tracking. By augmenting the system with the impairments due to hovering and rotational motion, we show that the proposed DQN algorithm can reform the beam in real-time with negligible error. It is demonstrated that the proposed DQN algorithm attains an exceptional performance improvement. We show that it requires a few iterations only for fine-tuning its parameters without observing any plateaus irrespective of the hovering tolerance.

Multiobjective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

Multi-UAV Trajectory Design and Power Control Based on Deep Reinforcement Learning.

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

Joint 3D Deployment and Power Allocation for UAV-BS: A Deep Reinforcement Learning Approach

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Joint Path planning and Power Allocation of a Cellular-Connected UAV using Apprenticeship Learning via Deep Inverse Reinforcement Learning

Distributed 3D-Beam Reforming for Hovering-Tolerant UAVs Communication over Coexistence: A Deep-Q Learning for Intelligent Space-Air-Ground Integrated Networks

Distributed UAV-BSs Trajectory Optimization for User-Level Fair Communication Service with Multi-Agent Deep Reinforcement Learning

Joint Multi-UAV Deployment and Resource Allocation Based on Personalized Federated Deep Reinforcement Learning.

Deep Reinforcement Learning-enabled Dynamic UAV Deployment and Power Control in Multi-UAV Wireless Networks

UAV-Assisted Enhanced Coverage and Capacity in Dynamic MU-mMIMO IoT Systems: A Deep Reinforcement Learning Approach

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks