Abstract:Unmanned aerial vehicles (UAVs) can provide flexible network coverage services. UAVs can be applied in a large number of scenarios, such as emergency communication and network access in areas without terrestrial network coverage. However, UAVs are limited to relatively short communication range and restricted energy resources. In extreme conditions such as disasters, there may also be a problem that the communication bandwidth is limited and the UAV cannot communicate with the server with a large amount of information, so a decentralized solution is expected. In addition, the interaction between multiple objectives and multiple UAVs leads to a huge state space, which makes large-scale practical applications difficult. To simplify complex interactions, we modeled the UAV control problem with mean-field game (MFG). We propose a new UAV control method, the mean-field trust region policy optimization (MFTRPO), which uses the MFG method to construct the Hamilton–Jacobi–Bellman/Fokker–Planck–Kolmogorov equation that obtains the optimal solution and solves the difficulties in the practical application through the trust region policy optimization and neural network feature embedding methods. The proposed method: 1) maximizes communication efficiency while ensuring fair communication range and network connectivity; 2) fuses the mean-field theory with deep reinforcement learning techniques; and 3) is scalable and adaptive. We conduct extensive simulations for performance evaluation. The simulation results have shown that MFTRPO significantly and consistently outperforms two commonly used baseline methods in terms of coverage, fairness, and energy consumption.

Flocking Control of UAV Swarms with Deep Reinforcement Leaming Approach

Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Oracle-Guided Deep Reinforcement Learning for Large-Scale Multi-UAVs Flocking and Navigation.

Attention-Based Population-Invariant Deep Reinforcement Learning for Collision-Free Flocking with A Scalable Fixed-Wing UAV Swarm.

Flocking of Under-Actuated Unmanned Surface Vehicles via Deep Reinforcement Learning and Model Predictive Path Integral Control

PPO-Exp: Keeping Fixed-Wing UAV Formation with Deep Reinforcement Learning

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

Collaborative Decision-making in Heterogeneous UAV Swarms Based on Multi-agent Deep Reinforcement Learning

Reinforcement Learning-Based Swarm Control for UAVs in Static and Dynamic Multi-Obstacle Environments

Multi-vehicle Flocking Control with Deep Deterministic Policy Gradient Method

UAV Visual Flight Control Method Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Flocking Motion of Multi-UAV systems: Learn from a Digital Twin

PASCAL: PopulAtion-Specific Curriculum-based MADRL for collision-free flocking with large-scale fixed-wing UAV swarms

Decentralized Learning Control for Multi-UAV Swarm Simultaneous Coverage and Tracking

A Multi-agent Deep Reinforcement Learning Method for UAVs Cooperative Pursuit Problem

Joint Communication and Action Learning in Multi-Target Tracking of UAV Swarms with Deep Reinforcement Learning

Deep Reinforcement Learning-Driven Collaborative Rounding-Up for Multiple Unmanned Aerial Vehicles in Obstacle Environments

Mean Field Deep Reinforcement Learning for Fair and Efficient UAV Control

Multi-UAV Flocking Control with a Hierarchical Collective Behavior Pattern Inspired by Sheep

Collision-Avoiding Flocking With Multiple Fixed-Wing UAVs in Obstacle-Cluttered Environments: A Task-Specific Curriculum- Based MADRL Approach