Abstract:Notwithstanding the successful design of state-of-the-art cooperative control protocols to accomplish formation tracking for multiple unmanned aerial vehicles (UAVs), the assurance of performance optimality cannot be guaranteed in the face of complex disturbances affecting these multi-UAV systems. In order to surmount this challenge, this research endeavor aims to establish a feedforward-feedback learning-based optimal control methodology to facilitate cooperative UAV formation tracking in the presence of intricate disturbances. To be more precise, by leveraging backstepping-based feedback control, the problem of UAV formation tracking is transformed into an equivalent optimal regulation problem. Consequently, a learning-based feedforward control scheme is devised, wherein the cooperative policy iteration algorithm is formulated based on a two-player zero-sum game. The critic-only echo state network (ESN) is employed to approximate the optimal feedforward control policies, with the inclusion of an online adaptive tuning law and compensation terms to alleviate the persistence of excitation condition and eliminate the need for an initial admissible control. As a result, the closed-loop stability is guaranteed in terms of uniformly ultimately boundedness for tracking errors and ESN weights. Note to Practitioners—In real-world scenarios, the flight of multiple UAVs is invariably affected by intricate disturbances, resulting in compromised tracking precision. There is an urgent need to enhance resistance to disturbances and ensure optimal performance for cooperative formation tracking of multiple UAVs. Beyond the capabilities of model-based controllers, the integration of reinforcement learning has shown promise in achieving robust control actions. By introducing the cooperative policy iteration algorithm based on a two-player zero-sum game, the tracking performances of UAV formation can be further optimized. In order to facilitate the practical application of reinforcement learning in UAV systems, our proposed algorithm addresses the persistency of excitation condition by incorporating innovative compensation terms into the ESN tuning law. Furthermore, we resolve the requirement for initial admissible control by introducing a novel piecewise compensation term into the ESN tuning law, which is based on a newly proposed Lyapunov function.

Formation cooperative trajectory tracking control for unmanned aerial vehicles via differential game and reinforcement learning

Formation Control of Multi-UAV Systems with Time and Angle Constraints

Learning-Based Optimal Cooperative Formation Tracking Control for Multiple UAVs: A Feedforward-Feedback Design Framework

Human-in-the-loop Distributed Cooperative Tracking Control with Applications to Autonomous Ground Vehicles: A Data-Driven Mixed Iteration Approach

Formation Control of Multi-UAV Systems Using Zeroing Dynamics Method

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Switching Learning-Based Cooperative Control with Its Application to Connected Automated Vehicles

Optimal Formation Control and Visual Simulation for Multiple Unmanned Aerial Vehicles (Uavs)

Decentralized Cooperative Control for Multivehicle Formation Without Velocity Measurement

Extended Kalman Filter Based Resilient Formation Tracking Control of Multiple Unmanned Vehicles via Game-Theoretical Reinforcement Learning

Distributed MPC for Trajectory Tracking and Formation Control of Multi-UAVs With Leader-Follower Structure

Vision-Based Formation Control Of A Heterogeneous Unmanned System

Cooperative deterministic learning and formation control for underactuated USVs with prescribed performance

Integrated Optimal Formation Control of Multiple Unmanned Aerial Vehicles

Optimal Unmanned Ground Vehicle—Unmanned Aerial Vehicle Formation-Maintenance Control for Air-Ground Cooperation

Leader-follower Formation Control of Multiple Vertical Takeoff and Landing UAVs: Distributed Estimator Design and Accurate Trajectory Tracking

Robust finite-time cooperative formation control of UGV-UAV with model uncertainties and actuator faults

A Generic Formation Controller and State Observer for Multiple Unmanned Systems

A Consensus-Based Model Predictive Control with Optimized Line-of-Sight Guidance for Formation Trajectory Tracking of Autonomous Underwater Vehicles

Coordinated Transportation of Tethered Multi-rotor UAVs Based on Differential Graphical Games