Abstract:In this study, a novel online adaptive dynamic programming (ADP)-based algorithm is developed for solving the optimal control problem of affine non-linear continuous-time systems with unknown internal dynamics. The present algorithm employs an observer-critic architecture to approximate the Hamilton-Jacobi-Bellman equation. Two neural networks (NNs) are used in this architecture: an NN state observer is constructed to estimate the unknown system dynamics and a critic NN is designed to derive the optimal control instead of typical action-critic dual networks employed in traditional ADP algorithms. Based on the developed architecture, the observer NN and the critic NN are tuned simultaneously. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees stability of the closed-loop system. No initial stabilising control is required, and by using recorded and instantaneous data simultaneously for the adaptation of the critic, the restrictive persistence of excitation condition is relaxed. In addition, Lyapunov direct method is utilised to demonstrate the uniform ultimate boundedness of the weights of the observer NN and the critic NN. Finally, an example is provided to verify the effectiveness of the present approach.

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Output Feedback Adaptive Robust Learning Control of a Class of Nonlinear Systems with Periodic Disturbances

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Reinforcement-Learning-Based Controller Design for Nonaffine Nonlinear Systems.

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

ℒ1adaptive Controller for a Class of Non-Affine Multi-Input Multi-Output Nonlinear Systems

Reinforcement Learning-Based Control for a Class of Nonlinear Systems with Unknown Control Directions

Discrete-time Online Learning Control for a Class of Unknown Nonaffine Nonlinear Systems Using Reinforcement Learning.

State and Input Constrained Output-Feedback Adaptive Optimal Control of Affine Nonlinear Systems

Adaptive critic-based tracking control of non-affine nonlinear discrete-time systems with unknown dynamics

Observer-Based Adaptive Output Feedback Control for Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement Learning.

Online Adaptive Controller Design Of Partially Unknown Nonlinear Systems With State Time-Delay Via Actor-Critic Architecture

Online Approximate Optimal Control for Affine Non-Linear Systems with Unknown Internal Dynamics Using Adaptive Dynamic Programming

Model-free Nonlinear Robust Control Design Via Online Critic Learning.

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

State and Parameter Estimation for Affine Nonlinear Systems