Abstract:In this article, a model-free Q-learning algorithm is proposed to solve the tracking problem of linear discrete-time systems with completely unknown system dynamics. To eliminate tracking errors, a performance index of the Q-learning approach is formulated, which can transform the tracking problem into a regulation one. Compared with the existing adaptive dynamic programming (ADP) methods and Q-learning approaches, the proposed performance index adds a product term composed of a gain matrix and the reference tracking trajectory to the control input quadratic form. In addition, without requiring any prior knowledge of the dynamics of the original controlled system and command generator, the control policy obtained by the proposed approach can be deduced by an iterative technique relying on the online information of the system state, the control input, and the reference tracking trajectory. In each iteration of the proposed method, the desired control input can be updated by the iterative criteria derived from a precondition of the controlled system and the reference tracking trajectory, which ensures that the obtained control policy can eliminate tracking errors in theory. Moreover, to effectively use less data to obtain the optimal control policy, the off-policy approach is introduced into the proposed algorithm. Finally, the effectiveness of the proposed algorithm is verified by a numerical simulation.

An ADDHP-based Q-learning Algorithm for Optimal Tracking Control of Linear Discrete-Time Systems with Unknown Dynamics

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Adaptive Dynamic Programming for Discrete-Time LQR Optimal Tracking Control Problems with Unknown Dynamics

Linear Quadratic Tracking Control of Unknown Systems: A Two-Phase Reinforcement Learning Method.

Model-Free Q-Learning for the Tracking Problem of Linear Discrete-Time Systems

An Optimal Tracking Control Method with Q-learning for Discrete-time Linear Switched System

Linear Quadratic Tracking Control of Unknown Discrete-Time Systems Using Value Iteration Algorithm

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Quadratic Tracking Control of Linear Stochastic Systems with Unknown Dynamics Using Average Off-Policy Q-Learning Method

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Adaptive Learning-Based Path-Tracking Control for Unknown Vehicle Systems under Performance Optimization

Iterative Q-learning-based Nonlinear Optimal Tracking Control

Data-driven finite-horizon optimal tracking control scheme for completely unknown discrete-time nonlinear systems.

Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Based on Q-Learning Optimal Tracking Control Schemes for Linear Itô Stochastic Systems with Markovian Jumps

Data-driven Optimal Tracking Control for Discrete-Time Systems with Delays Using Adaptive Dynamic Programming.

Online Optimal Tracking Control of Continuous-Time Linear Systems with Unknown Dynamics by Using Adaptive Dynamic Programming

Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method

Data-Based Optimal Tracking Control of Nonaffine Nonlinear Discrete-Time Systems.

Robust Adaptive Quadratic Tracking Control of Continuous-Time Linear Systems with Unknown Dynamics.

Data-Driven Tracking Control for Multi-Agent Systems with Unknown Dynamics Via Multithreading Iterative Q-Learning