Abstract:Exploiting highly maneuverable unmanned aerial vehicles (UAVs) has been considered as an efficient way to assist wireless systems, e.g., for applications of data collection. However, several challenges remain to be addressed in the design of such UAV-assisted networks, including multi-UAV joint trajectory determination, data privacy protection, and adaption to the complex channel environment particularly with mobile ground devices (GDs). In this paper, we study a multi-UAV assisted data collection system where UAVs collect data locally from mobile GDs. The aim is to minimize the whole operation time cost via jointly optimizing the UAVs' three-dimensional (3D) trajectory together with the GDs' communication scheduling, while satisfying the constraints of no-fly zones (NFZs) and collision avoidance. With a nonconvex feasible set (due to the NFZs), the established problem is nonconvex. Moreover, the randomness of GDs movements significantly reduces the performance of a typical redesign, i.e., determining the UAVs' trajectory and users' scheduling before starting the data collection task. To tackle these issues, we first transform the established problem into a Markov decision one, and then propose a multi-agent federated reinforcement learning (MAFRL)-based approach to optimize the dynamic long-term objective via jointly determining UAVs' 3D trajectory and GD's communication scheduling. A multi-step propagation technique and a dueling network architecture are adopted to enhance the neural network utilized to train agents, i.e., to accelerate the convergence rate of the proposed method and improve its overall stability. Finally, experimental results reveal the effectiveness of our proposed method in the considered practical scenario.

Deep Reinforcement Learning Based Data Collection with Charging Stations

Research on the UAV-aided Data Collection and Trajectory Design Based on the Deep Reinforcement Learning

Energy-Efficient UAV-Enabled Data Collection via Wireless Charging: A Reinforcement Learning Approach

Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach

Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learning

Multi-Task Transfer Deep Reinforcement Learning for Timely Data Collection in Rechargeable-UAV-aided IoT Networks

Learning-Based Aerial Charging Scheduling for UAV-Based Data Collection

Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks

DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Multi-UAV WRSN charging path planning based on improved heed and IA-DRL

UAVs and Mobile Sensors Trajectories Optimization with Deep Learning Trained by Genetic Algorithm Towards Data Collection Scenario

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning

Federated deep reinforcement learning based trajectory design for UAV-assisted networks with mobile ground devices

Deep Reinforcement Learning for Online Routing of Unmanned Aerial Vehicles with Wireless Power Transfer

Towards Real-Time Path Planning through Deep Reinforcement Learning for a UAV in Dynamic Environments

Rechargeable UAV Trajectory Optimization for Real-Time Persistent Data Collection of Large-Scale Sensor Networks

A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning