Abstract:Unmanned Aerial Vehicles (UAVs) can be deployed as aerial wireless base stations which dynamically cover the wireless communication networks for Ground Users (GUs). The most challenging problem is how to control multi-UAVs to achieve on-demand coverage of wireless communication networks while maintaining connectivity among them. In this paper, the cooperative trajectory optimization of UAVs is studied to maximize the communication efficiency in the dynamic deployment of UAVs for emergency communication scenarios. We transform the problem into a Markov game problem and propose a distributed trajectory optimization algorithm, Double-Stream Attention multi-agent Actor-Critic (DSAAC), based on Multi-Agent Deep Reinforcement Learning (MADRL). The throughput, safety distance, and power consumption of UAVs are comprehensively taken into account for designing a practical reward function. For complex emergency communication scenarios, we design a double data stream network structure that provides a capacity for the Actor network to process state changes. Thus, UAVs can sense the movement trends of the GUs as well as other UAVs. To establish effective cooperation strategies for UAVs, we develop a hierarchical multi-head attention encoder in the Critic network. This encoder can reduce the redundant information through the attention mechanism, which resolves the problem of the curse of dimensionality as the number of both UAVs and GUs increases. We construct a simulation environment for emergency networks with multi-UAVs and compare the effects of the different numbers of GUs and UAVs on algorithms. The DSAAC algorithm improves communication efficiency by 56.7%, throughput by 71.2%, energy saving by 19.8%, and reduces the number of crashes by 57.7%.

An Application of Model Reference Adaptive Control for Multi-Agent Synchronization in Drone Networks

Neighbour-based Synchronous Distributed Receding Horizon Control for Multi-agent Systems

Distributed Synchronization Control of Nonaffine Multiagent Systems with Guaranteed Performance

Observer-Based Asynchronous Communication and Adaptive Control Protocols with Intermittent Updating and Interaction.

Neural Adaptive Distributed Formation Control of Nonlinear Multi-UAVs With Unmodeled Dynamics

Multi-Agent Spatial Predictive Control with Application to Drone Flocking (Extended Version)

Reinforcement Learning Driven Cooperative Ball Balance in Rigidly Coupled Drones

Control and real-time experiments for a multi-agent aerial transportation system

Synchronization tracking control of networked multi-axis motion systems: A cooperative distributed model predictive control approach

CoNi-MPC: Cooperative Non-inertial Frame Based Model Predictive Control

Robust synchronization and policy adaptation for networked heterogeneous agents

Continuous Maneuver Control and Data Capture Scheduling of Autonomous Drone in Wireless Sensor Networks

Multilayer Neural Dynamics-Based Adaptive Control of Multirotor UAVs for Tracking Time-Varying Tasks

Nonlinear Model Predictive Control for Multi-Micro Aerial Vehicle Robust Collision Avoidance

Energy-Aware Multi-Agent Reinforcement Learning for Collaborative Execution in Mission-Oriented Drone Networks

A robust adaptive formation control methodology for networked multi-UAV systems with applications to cooperative payload transportation

Robust Control Strategy for Quadrotor Drone Using Reference Model-Based Deep Deterministic Policy Gradient

Deep Nonlinear Adaptive Control for Unmanned Aerial Systems Operating under Dynamic Uncertainties

A Novel Neural Network-Based Robust Adaptive Formation Control for Cooperative Transport of a Payload Using Two Underactuated Quadcopters

Energy-Efficient Multi-UAVs Cooperative Trajectory Optimization for Communication Coverage: An MADRL Approach

Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimization