Abstract:Summary This article introduces a novel optimal trajectory tracking control scheme designed for uncertain linear discrete‐time (DT) systems. In contrast to traditional tracking control methods, our approach removes the requirement for the reference trajectory to align with the generator dynamics of an autonomous dynamical system. Moreover, it does not demand the complete desired trajectory to be known in advance, whether through the generator model or any other means. Instead, our approach can dynamically incorporate segments (finite horizons) of reference trajectories and autonomously learn an optimal control policy to track them in real time. To achieve this, we address the tracking problem by learning a time‐varying ‐function through state feedback. This ‐function is then utilized to calculate the optimal feedback gain and explicitly time‐varying feedforward control input, all without the need for prior knowledge of the system dynamics or having the complete reference trajectory in advance. Additionally, we introduce an adaptive observer to extend the applicability of the tracking control scheme to situations where full state measurements are unavailable. We rigorously establish the closed‐loop stability of our optimal adaptive control approach, both with and without the adaptive observer, employing Lyapunov theory. Moreover, we characterize the optimality of the controller with respect to the finite horizon length of the known components of the desired trajectory. To further enhance the controller's adaptability and effectiveness in multitask environments, we employ the Efficient Lifelong Learning Algorithm, which leverages a shared knowledge base within the recursive least squares algorithm for multitask ‐learning. The efficacy of our approach is substantiated through a comprehensive set of simulation results by using a power system example.

Optimal State Tracking Control for Linear Discrete-time Systems Via Value Iteration

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Linear Quadratic Tracking Control of Unknown Discrete-Time Systems Using Value Iteration Algorithm

Model-Free Optimal Tracking Design With Evolving Control Strategies via Q-Learning

Iterative Q-learning-based Nonlinear Optimal Tracking Control

H∞ Tracking Control for Linear Discrete‐time Systems Via Reinforcement Learning

Data-Based Optimal Tracking Control of Nonaffine Nonlinear Discrete-Time Systems.

General value iteration based reinforcement learning for solving optimal tracking control problem of continuous-time affine nonlinear systems.

Learning-based adaptive optimal control of linear time-delay systems: A value iteration approach

An Optimal Tracking Control Method with Q-learning for Discrete-time Linear Switched System

Data-based Optimal Tracking of Autonomous Nonlinear Switching Systems.

Optimal control for discrete-time affine non-linear systems using general value iteration

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Optimal Tracking Control Problem for Linear State and Control Time-Delay Systems

Optimal output tracking control for discrete-time systems with delayed state and input

Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Optimal tracking control of batch processes with time-invariant state delay: Adaptive Q-learning with two-dimensional state and control policy

Optimal trajectory tracking for uncertain linear discrete‐time systems using time‐varying Q‐learning

Approximate Design of Optimal Tracking Controller for Systems with Delayed State and Control

Optimal Tracking Control for Linear Discrete-Time Stochastic System Based on Adaptive Dynamic Programming

Value Approximator-based Learning Model Predictive Control for Iterative Tasks