Abstract:This article is concerned with the optimal tracking control problem of the coupled Markov jump system (CMJS) by using the reinforcement learning (RL) technique. Based on the conventional optimal tracking architecture, an offline tracking iteration algorithm is first designed to solve the coupled algebraic Riccati equation that can hardly he solved by mathematical methods directly. To overcome the crucial requirements and existing shortcomings in the offline tracking method, a novel integral RL (IRL) tracking algorithm is first proposed for CMJS, which develops a transition-probability-free optimal tracking control scheme with a reconstructed augmented system and discounted cost function. Both the requirements of transition probability pi(ij) and system matrix A(i) are avoided via the designed IRI, algorithm. The stability and convergence of the novel schemes are proved by the Lyapunov theory, and the tracking objective is achieved as desired. Finally, we apply the designed algorithms in a fourth-order Markov jump control problem and the stochastic mass, spring, and damper system to track continuous sinusoidal waveforms, and the simulation results are provided to show the effectiveness and applicability. Note to Practitioners-In the practical engineering systems, many useful signals and interference vary randomly. Therefore, the tracking control of stochastic systems and dynamics, such as the Markovion, Ito's, Wiener, and Martingale processes, plays an important role in the modern industry. As a matter of fact, it is always desired to reduce the requirement of exact information and transition probability in the homogeneous Markovian process, which is very difficult to obtain accurate measurements. One way is integrating the adaptive reinforcement learning (RI) technique into the Markovian systems to learn this implicit information. However, a major restriction of the RL technique is that the control policy should be related to the finite performance index, which generally invalidates the optimal tracking solutions. In order to tackle this difficulty, by designing a novel parallel scheme via integral RL (IRL) technique, the solution of the coupled algebraic Riccati equation is solved, and the transition probability can be completely unknown during the learning process.

Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Asynchronous iterative Q-learning based tracking control for nonlinear discrete-time multi-agent systems

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Reinforcement Learning-Based Optimal Tracking Control for Uncertain Multi-Agent Systems with Uncertain Topological Networks

Distributed Optimal Control of Nonlinear Multi‐agent Systems Based on Integral Reinforcement Learning

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Differential graphical game‐based multi‐agent tracking control using integral reinforcement learning

Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach

Practical Optimized Time-Varying Formation Tracking Control for Nonlinear Multi-Agent Systems with Unknown Dynamics and Non-Cooperative Leader

Q-learning Solution for Optimal Consensus Control of Discrete-Time Multiagent Systems Using Reinforcement Learning

Approximated Optimal Consensus Tracking Control of Non-affine Nonlinear Multi-agent Systems

Model-Free Optimal Tracking Control Via Critic-Only Q-Learning.

Optimal couple-group tracking control for the heterogeneous multi-agent systems with cooperative-competitive interactions via reinforcement learning method

Data-Driven Tracking Control for Multi-Agent Systems with Unknown Dynamics Via Multithreading Iterative Q-Learning

Optimal Tracking Control of Heterogeneous MASs Using Event-Driven Adaptive Observer and Reinforcement Learning

Practical Optimal Formation-Containment Tracking Control of Nonlinear Multiagent Systems With Unknown Dynamics

Parallel Optimal Tracking Control Schemes for Mode-Dependent Control of Coupled Markov Jump Systems Via Integral RL Method

Optimal tracking agent: a new framework of reinforcement learning for multiagent systems.

Optimal Tracking Control of Mechatronic Servo System Using Integral Reinforcement Learning