Abstract:In this article, a novel continuous-time optimal tracking controller is proposed for the single-input-single-output linear system with completely unknown dynamics. Unlike those existing solutions to the optimal tracking control problem, the proposed controller introduces an integral compensation to reduce the steady-state error and regulates the feedforward part simultaneously with the feedback part. An augmented system composed of the integral compensation, error dynamics, and desired trajectory is established to formulate the optimal tracking control problem. The input energy and tracking error of the optimal controller are minimized according to the objective function in the infinite horizon. With the application of reinforcement learning techniques, the proposed controller does not require any prior knowledge of the system drift or input dynamics. The integral reinforcement learning method is employed to approximate the Q-function and update the critic network on-line. And the actor network is updated with the deterministic learning method. The Lyapunov stability is proved under the persistence of excitation condition. A case study on a hydraulic loading system has shown the effectiveness of the proposed controller by simulation and experiment.

Learning Optimal Control Policy for Unknown Discrete-Time Systems

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Model-free Optimal Control of Discrete-Time Systems with Additive and Multiplicative Noises

Discrete-Time Adaptive Iterative Learning Control for High-Order Nonlinear Systems with Unknown Control Directions

Generalized Policy Iteration-based Reinforcement Learning Algorithm for Optimal Control of Unknown Discrete-time Systems

Data-Based Efficient Off-Policy Stabilizing Optimal Control Algorithms for Discrete-Time Linear Systems via Damping Coefficients

Adaptive Optimal Control for a Class of Continuous-Time Affine Nonlinear Systems with Unknown Internal Dynamics

Data-based Optimal Control for Discrete-time Systems Via Deep Deterministic Policy Gradient Adaptive Dynamic Programming

Optimal Tracking Control for Non-Zero-sum Games of Linear Discrete-Time Systems Via Off-Policy Reinforcement Learning

Bias-policy iteration based optimal control for unknown continuous-time linear periodic systems

Model-Free Control of Time-Delay Systems Via Policy Gradient Based Adaptive Learning Algorithm

Optimal control of nonlinear system based on deterministic policy gradient with eligibility traces

Optimal Learning Control Scheme for Discrete-Time Systems with Nonuniform Trials.

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

Data-driven Adaptive Optimal Control for Discrete-Time Linear Time-Invariant Systems

Policy Optimization Adaptive Dynamic Programming for Optimal Control of Input-Affine Discrete-Time Nonlinear Systems.

Policy Gradient Reinforcement Learning for Parameterized Continuous-Time Optimal Control

Data-Driven H∞ Optimal Output Feedback Control for Linear Discrete-Time Systems Based on Off-Policy Q-Learning

Average Cost Optimal Control of Stochastic Systems Using Reinforcement Learning

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Adaptive Optimal Control of Discrete-Time Linear Systems with Discounted Value: Off-Policy Reinforcement Learning