Abstract:In this paper, we present a novel distributed UAVs beam reforming approach to dynamically form and reform a space-selective beam path in addressing the coexistence with satellite and terrestrial communications. Despite the unique advantage to support wider coverage in UAV-enabled cellular communications, the challenges reside in the array responses' sensitivity to random rotational motion and the hovering nature of the UAVs. A model-free reinforcement learning (RL) based unified UAV beam selection and tracking approach is presented to effectively realize the dynamic distributed and collaborative beamforming. The combined impact of the UAVs' hovering and rotational motions is considered while addressing the impairment due to the interference from the orbiting satellites and neighboring networks. The main objectives of this work are two-fold: first, to acquire the channel awareness to uncover its impairments; second, to overcome the beam distortion to meet the quality of service (QoS) requirements. To overcome the impact of the interference and to maximize the beamforming gain, we define and apply a new optimal UAV selection algorithm based on the brute force criteria. Results demonstrate that the detrimental effects of the channel fading and the interference from the orbiting satellites and neighboring networks can be overcome using the proposed approach. Subsequently, an RL algorithm based on Deep Q-Network (DQN) is developed for real-time beam tracking. By augmenting the system with the impairments due to hovering and rotational motion, we show that the proposed DQN algorithm can reform the beam in real-time with negligible error. It is demonstrated that the proposed DQN algorithm attains an exceptional performance improvement. We show that it requires a few iterations only for fine-tuning its parameters without observing any plateaus irrespective of the hovering tolerance.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges faced in wireless communication using unmanned aerial vehicles (UAVs) in non - terrestrial networks (NTN) and space - air - ground integrated networks (SAGIN), especially in the Sub - 6 GHz frequency band range. Specifically, the paper focuses on the following points: 1. **Dynamic Beamforming**: The paper proposes a new distributed UAV beam reconstruction method to dynamically form and reconstruct spatially selective beam paths, solving the problem of coexistence with satellite and ground communications. 2. **Hover Tolerance**: Due to the random rotational movement and hovering characteristics of UAVs, antenna gain mismatch occurs and the target coverage area changes frequently. Therefore, a flexible beamforming technique with hover tolerance is required. 3. **Interference Suppression**: Considering the influence of UAV hovering and rotational movement on the channel, as well as the interference from orbital satellites and neighboring networks. The goal of the paper is to obtain channel awareness between UAVs and user equipment (UE), identify spatial interference, and determine the optimal link to avoid interference. 4. **Beam Distortion Correction**: By configuring selected UAVs for beam reconstruction to meet the quality of service (QoS) requirements. The paper studies the change in the angle of arrival and points out that due to the random hovering and rotational movement of UAVs, the beam cannot be accurately pointed at the receiver. 5. **Optimization Algorithm**: A new optimal UAV selection algorithm based on the brute - force search criterion is defined and applied. This algorithm can detect the best UAV for beamforming based on its direction and channel conditions. 6. **Real - Time Beam Tracking**: A reinforcement learning algorithm based on the deep Q - network (DQN) is developed for real - time beam tracking. A deep neural network (DNN) is used as an approximation function to estimate the Q - value, and the DQN agent learns from experience through experience replay. Overall, the paper aims to solve the key problems faced by UAVs in efficient and reliable communication in space - air - ground integrated networks through deep learning and reinforcement learning techniques, especially maintaining high - quality communication performance in dynamic environments and interference conditions.

Distributed 3D-Beam Reforming for Hovering-Tolerant UAVs Communication over Coexistence: A Deep-Q Learning for Intelligent Space-Air-Ground Integrated Networks

Learning-Enabled Radar-Assisted Predictive Beamforming for UAV-Aided Networks

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

Deep Reinforcement Learning for Beam Management in UAV Relay mmWave Networks

3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Fairness-Driven Optimization of RIS-Augmented 5G Networks for Seamless 3D UAV Connectivity Using DRL Algorithms

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

Machine-Learning Beam Tracking and Weight Optimization for mmWave Multi-UAV Links

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Quantum-Assisted Adaptive Beamforming in UASs Network: Enhancing Airborne Communication via Collaborative UASs for NextG IoT

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Location-aware Predictive Beamforming for UAV Communications: A Deep Learning Approach

Deep Reinforcement Learning for Interference Management in UAV-based 3D Networks: Potentials and Challenges

Learning Based Dynamic Cluster Reconfiguration for UAV Mobility Management with 3D Beamforming

QoE Optimization for Live Video Streaming in UAV-to-UAV Communications via Deep Reinforcement Learning

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks