Abstract:This paper investigates a Q-learning scheme for the optimal consensus control of discrete-time multiagent systems. The Q-learning algorithm is conducted by reinforcement learning (RL) using system data instead of system dynamics information. In the multiagent systems, the agents are interacted with each other and at least one agent can communicate with the leader directly, which is described by an algebraic graph structure. The objective is to make all the agents achieve synchronization with leader and make the performance indices reach Nash equilibrium. On one hand, the solutions of the optimal consensus control for multiagent systems are acquired by solving the coupled Hamilton–Jacobi–Bellman (HJB) equation. However, it is difficult to get analytical solutions directly of the discrete-time HJB equation. On the other hand, accurate mathematical models of most systems in real world are hard to be obtained. To overcome these difficulties, Q-learning algorithm is developed using system data rather than the accurate system model. We formulate performance index and corresponding Bellman equation of each agent i. Then, the Q-function Bellman equation is acquired on the basis of Q-function. Policy iteration is adopted to calculate the optimal control iteratively, and least square (LS) method is employed to motivate the implementation process. Stability analysis of proposed Q-learning algorithm for multiagent systems by policy iteration is given. Two simulation examples are experimented to verify the effectiveness of the proposed scheme.

Data-Driven Optimal Bipartite Consensus Control for Second-Order Multiagent Systems via Policy Gradient Reinforcement Learning

Optimal Bipartite Consensus Control for Heterogeneous Unknown Multi-Agent Systems Via Reinforcement Learning

A Novel Optimal Bipartite Consensus Control Scheme for Unknown Multi-Agent Systems Via Model-Free Reinforcement Learning.

Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning

Adaptive Optimal Bipartite Consensus Control for Heterogeneous Multi-Agent Systems

Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method

Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach

RL-Based Adaptive Optimal Bipartite Consensus Control for Nonlinear Heterogeneous MASs Via Event-Triggered State Feedback

Learning-Based Robust Bipartite Consensus Control for a Class of Multiagent Systems

Observer-Based Consensus Control for MASs With Prescribed Constraints via Reinforcement Learning Algorithm

Output-feedback Optimized Consensus for Directed Graph Multi-Agent Systems Based on Reinforcement Learning and Subsystem Error Derivatives.

Data-driven Distributed Output Consensus Control for Multi-Agent Systems with Unknown Internal State

Distributed Optimal Consensus Control for Nonlinear Multiagent System with Unknown Dynamic.

Optimal Antisynchronization Control for Unknown Multiagent Systems with Deep Deterministic Policy Gradient Approach.

Data-driven output consensus for a class of discrete-time multiagent systems by reinforcement learning techniques

Q-learning Solution for Optimal Consensus Control of Discrete-Time Multiagent Systems Using Reinforcement Learning

Data-Driven Optimal Consensus Control for Switching Multiagent Systems Via Joint Communication Graph

Distributed Model Predictive Control for Optimal Consensus of Constrained Multi-Agent Systems

Distributed Optimal Consensus Control of Constrained Multiagent Systems: A Nonseparable Optimization Perspective

Optimal Consensus Control for Heterogeneous Nonlinear Multiagent Systems with Partially Unknown Dynamics

Data-Driven Distributed Output Consensus Control for Partially Observable Multiagent Systems.