Abstract:In this article, a model-free Q-learning algorithm is proposed to solve the tracking problem of linear discrete-time systems with completely unknown system dynamics. To eliminate tracking errors, a performance index of the Q-learning approach is formulated, which can transform the tracking problem into a regulation one. Compared with the existing adaptive dynamic programming (ADP) methods and Q-learning approaches, the proposed performance index adds a product term composed of a gain matrix and the reference tracking trajectory to the control input quadratic form. In addition, without requiring any prior knowledge of the dynamics of the original controlled system and command generator, the control policy obtained by the proposed approach can be deduced by an iterative technique relying on the online information of the system state, the control input, and the reference tracking trajectory. In each iteration of the proposed method, the desired control input can be updated by the iterative criteria derived from a precondition of the controlled system and the reference tracking trajectory, which ensures that the obtained control policy can eliminate tracking errors in theory. Moreover, to effectively use less data to obtain the optimal control policy, the off-policy approach is introduced into the proposed algorithm. Finally, the effectiveness of the proposed algorithm is verified by a numerical simulation.

Off-Policy Reinforcement Learning for Tracking in Continuous-Time Systems on Two Time Scales

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Operational Optimal Tracking Control for Industrial Multirate Systems Subject to Unknown Disturbances

Reinforcement learning for optimal tracking of large-scale systems with multitime scales

Reinforcement Learning for Input Constrained Sub-optimal Tracking Control in Discrete-time Two-time-scale Systems

Linear Quadratic Tracking Control of Unknown Systems: A Two-Phase Reinforcement Learning Method.

Optimal Tracking Control for Non-Zero-sum Games of Linear Discrete-Time Systems Via Off-Policy Reinforcement Learning

Off-policy integral reinforcement learning optimal tracking control for continuous-time chaotic systems

New Methods for Optimal Operational Control of Industrial Processes Using Reinforcement Learning on Two Time Scales

Model-Free Q-Learning for the Tracking Problem of Linear Discrete-Time Systems

Data-driven Optimal Tracking Control for a Class of Affine Non-Linear Continuous-Time Systems with Completely Unknown Dynamics

Reinforcement Learning Tracking Control for Unknown Continuous Dynamic Systems

Off-Policy Reinforcement Learning for Optimal Preview Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics

Iterative Q-learning-based Nonlinear Optimal Tracking Control

Off-policy reinforcement learning for tracking control of discrete-time Markov jump linear systems with completely unknown dynamics.

Off-Policy Reinforcement Learning: Optimal Operational Control for Two-Time-Scale Industrial Processes.

Adaptive Dynamic Programming for Tracking Design of Uncertain Nonlinear Systems with Disturbances and Input Constraints

Finite horizon optimal tracking control of partially unknown linear continuous-time systems using policy iteration

Off-policy two-dimensional reinforcement learning for optimal tracking control of batch processes with network-induced dropout and disturbances

Robust Output Regulation and Reinforcement Learning-Based Output Tracking Design for Unknown Linear Discrete-Time Systems

Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System