Optimal Consensus Control for Discrete-Time Systems with State Delay Using Q-learning Solution.

Li Zhang,Shicheng Huo,Ya Zhang

DOI: https://doi.org/10.1109/icca54724.2022.9831830

2022-01-01

Abstract:In this paper, an optimal consensus control protocol based on Q-learning algorithm is proposed for a class of discrete-time multiagent systems with unknown system matrices and state delays. It is well known that coupled Hamilton Jacobi Bellman(HJB) equation is difficult to be solved especially for multiagent systems(MASs) with state delays. On the basis of the coordinate transformation, MASs with state delays can be converted to a corresponding delay-free system under certain conditions. Then, Q-learning algorithm is adopted by using the delay-free system data rather than the accurate system model. We formulate the Q-function Bellman equation and adopt policy iteration(PI) to calculate the optimal control iteratively. Finally, a simulation example is applied to demonstrate the validity of the optimal consensus control protocol.