Abstract:This paper investigates a Q-learning scheme for the optimal consensus control of discrete-time multiagent systems. The Q-learning algorithm is conducted by reinforcement learning (RL) using system data instead of system dynamics information. In the multiagent systems, the agents are interacted with each other and at least one agent can communicate with the leader directly, which is described by an algebraic graph structure. The objective is to make all the agents achieve synchronization with leader and make the performance indices reach Nash equilibrium. On one hand, the solutions of the optimal consensus control for multiagent systems are acquired by solving the coupled Hamilton–Jacobi–Bellman (HJB) equation. However, it is difficult to get analytical solutions directly of the discrete-time HJB equation. On the other hand, accurate mathematical models of most systems in real world are hard to be obtained. To overcome these difficulties, Q-learning algorithm is developed using system data rather than the accurate system model. We formulate performance index and corresponding Bellman equation of each agent i. Then, the Q-function Bellman equation is acquired on the basis of Q-function. Policy iteration is adopted to calculate the optimal control iteratively, and least square (LS) method is employed to motivate the implementation process. Stability analysis of proposed Q-learning algorithm for multiagent systems by policy iteration is given. Two simulation examples are experimented to verify the effectiveness of the proposed scheme.

General Second-Order Consensus of Discrete-Time Multiagent Systems Via Q-Learning Method

Distributed Reduced-Order Observer-Based Approach to Consensus Problems for Linear Multi-Agent Systems

Second-order consensus in multi-agent systems with directed topologies and communication constraints.

Leader-Follower Optimal Consensus of Discrete-Time Linear Multi-agent Systems based on Q-Learning

Second-Order Consensus for Multiagent Systems with Switched Dynamics

Q-learning Solution for Optimal Consensus Control of Discrete-Time Multiagent Systems Using Reinforcement Learning

Second-order Consensus of Discrete-Time Multi-Agent Systems Via One-Step Delayed Data

Consensus of Second-Order Multi-Agent Systems with Nonlinear Dynamics and Time Delay

A Game-theoretical Approach for a Finite-time Consensus of Second-order Multi-agent System

Consensus of second-order discrete-time multi-agent systems with fixed topology

Second-order Consensus of Multiagent Systems with Matrix-Weighted Network.

Second-order Consensus Seeking in Directed Networks of Multi-Agent Dynamical Systems Via Generalized Linear Local Interaction Protocols

Second-Order Consensus for Multiagent Systems with Directed Topologies and Nonlinear Dynamics

Consensus in Multi-Agent Systems with Second-Order Dynamics and Sampled Data

An Iterative Q-learning Based Global Consensus of Discrete-Time Saturated Multi-Agent Systems.

Consensus of Second-Order Discrete-Time Multi-Agent Systems with Nonuniform Time-Delays and Dynamically Changing Topologies

On the General Consensus Protocol in Multi-Agent Networks with Second-Order Dynamics and Sampled Data

Consensus of Second Order Discrete-Time Multi-Agent Systems with Switching Topology

Consensus Control for A Class of Second-Order Multi-Agent Systems: an Iterative Learning Approach

Adaptive Second-Order Consensus of Multi-Agent Systems with Heterogeneous Nonlinear Dynamics and Time-Varying Delays.

Consensus Analysis of Second-Order Multiagent Systems with General Topology and Time Delay.