Abstract:It is necessary to consider the robustness in the tracking problem, which can effectively suppress the external disturbance to ensure the tracking performance. Different from previous tracking control methods, considering the robustness, completely unknown nonlinear system dynamics and constrained controller, we propose a data-based echo state network (ESN) approximated algorithm for a class of robust tracking problems. First, the robust tracking control problem (RTCP) is transformed into the optimal control problem of the according nominal system by designing a elaborate value function. To obtain the optimal control policy, we have to solve a Hamilton-Jacobi-Bellman equation (HJBE) about the augmented nominal system. It is well-known that modelling the accurate dynamics for the practical engineering applications is usually difficult, so the model-free integral reinforcement learning (IRL) algorithm is used to learn the optimal control policy and performance function simultaneously by only using systems data. In this IRL algorithm, a reservoir computing based ESN is used to approximate the performance function and control input. Contrast to other neural networks, ESN need not consider the choice of activation function, which can greatly reduce the difficulty and effort of neural network structure design. The output weights of the ESNs are iteratively updated towards the optimal ones by using least square algorithm and the pre-collected off-line system data. Then, using the converged output weights and ESNs, the tracking control input can be derived without knowing any system dynamic information. Finally, we demonstrate that the given system can be controlled to track the desired trajectory well under the proposed method by using two simulation examples.

Reinforcement Learning for Input Constrained Sub-optimal Tracking Control in Discrete-time Two-time-scale Systems

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Fixed-Time Tracking Control for State-Constrained Nonstrict-Feedback Systems Without Feasibility Conditions

Reinforcement learning for optimal tracking of large-scale systems with multitime scales

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System

Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Fuzzy Adaptive Tracking of Constrained Nonlinear Systems with Event-Sampling Reinforcement Learning

Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning

Adaptive output feedback reinforcement learning control for continuous time switched stochastic nonlinear systems with unknown control coefficients and full-state constraints

Reinforcement Learning Tracking Control for Unknown Continuous Dynamic Systems

Robust Output Regulation and Reinforcement Learning-Based Output Tracking Design for Unknown Linear Discrete-Time Systems

Model-Free Optimal Tracking Design With Evolving Control Strategies via Q-Learning

Reinforcement learning-based finite-time tracking control of an unknown unmanned surface vehicle with input constraints

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Signal Temporal Logic Neural Predictive Control

Optimized tracking control using reinforcement learning and backstepping technique for canonical nonlinear unknown dynamic system

Integral reinforcement learning‐based optimal tracking control for uncertain nonlinear systems under input constraint and specified performance constraints

Echo State Network-Based Robust Tracking Control for Unknown Constrained Nonlinear Systems by Using Integral Reinforcement Learning