Abstract:Notwithstanding the successful design of state-of-the-art cooperative control protocols to accomplish formation tracking for multiple unmanned aerial vehicles (UAVs), the assurance of performance optimality cannot be guaranteed in the face of complex disturbances affecting these multi-UAV systems. In order to surmount this challenge, this research endeavor aims to establish a feedforward-feedback learning-based optimal control methodology to facilitate cooperative UAV formation tracking in the presence of intricate disturbances. To be more precise, by leveraging backstepping-based feedback control, the problem of UAV formation tracking is transformed into an equivalent optimal regulation problem. Consequently, a learning-based feedforward control scheme is devised, wherein the cooperative policy iteration algorithm is formulated based on a two-player zero-sum game. The critic-only echo state network (ESN) is employed to approximate the optimal feedforward control policies, with the inclusion of an online adaptive tuning law and compensation terms to alleviate the persistence of excitation condition and eliminate the need for an initial admissible control. As a result, the closed-loop stability is guaranteed in terms of uniformly ultimately boundedness for tracking errors and ESN weights. Note to Practitioners—In real-world scenarios, the flight of multiple UAVs is invariably affected by intricate disturbances, resulting in compromised tracking precision. There is an urgent need to enhance resistance to disturbances and ensure optimal performance for cooperative formation tracking of multiple UAVs. Beyond the capabilities of model-based controllers, the integration of reinforcement learning has shown promise in achieving robust control actions. By introducing the cooperative policy iteration algorithm based on a two-player zero-sum game, the tracking performances of UAV formation can be further optimized. In order to facilitate the practical application of reinforcement learning in UAV systems, our proposed algorithm addresses the persistency of excitation condition by incorporating innovative compensation terms into the ESN tuning law. Furthermore, we resolve the requirement for initial admissible control by introducing a novel piecewise compensation term into the ESN tuning law, which is based on a newly proposed Lyapunov function.

Heterogeneous formation control of multiple UAVs with limited-input leader via reinforcement learning

Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

Cooperative Learning Formation Control of Multiple Autonomous Underwater Vehicles with Prescribed Performance Based on Position Estimation

Formation Control of Multi-UAV Systems with Time and Angle Constraints

Data-driven time-varying formation-containment control for a heterogeneous air-ground vehicle team subject to active leaders and switching topologies

Vision-Based Formation Control for a Heterogeneous Multi-Robot System

Leader–follower UAVs formation control based on a deep Q-network collaborative framework

Leader-follower formation control of mutilple UAVs based on ADRC:experiment research

Vision-Based Formation Control Of A Heterogeneous Unmanned System

Optimal Formation Control and Visual Simulation for Multiple Unmanned Aerial Vehicles (Uavs)

Multi-UAV Behavior-based Formation with Static and Dynamic Obstacles Avoidance via Reinforcement Learning

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Data-Driven Optimal Formation Control for Quadrotor Team With Unknown Dynamics

Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

Distributed Robust Formation Control of Heterogeneous Multi-UAVs With Disturbance Rejection

Learning-Based Optimal Cooperative Formation Tracking Control for Multiple UAVs: A Feedforward-Feedback Design Framework

Data-Driven Formation Control for Multiple Heterogeneous Vehicles in Air–Ground Coordination

Extended Kalman Filter Based Resilient Formation Tracking Control of Multiple Unmanned Vehicles via Game-Theoretical Reinforcement Learning

Distributed output formation tracking control of heterogeneous multi-agent systems using reinforcement learning

Distributed MPC for Trajectory Tracking and Formation Control of Multi-UAVs With Leader-Follower Structure