Abstract:In this study, a novel online adaptive dynamic programming (ADP)-based algorithm is developed for solving the optimal control problem of affine non-linear continuous-time systems with unknown internal dynamics. The present algorithm employs an observer-critic architecture to approximate the Hamilton-Jacobi-Bellman equation. Two neural networks (NNs) are used in this architecture: an NN state observer is constructed to estimate the unknown system dynamics and a critic NN is designed to derive the optimal control instead of typical action-critic dual networks employed in traditional ADP algorithms. Based on the developed architecture, the observer NN and the critic NN are tuned simultaneously. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees stability of the closed-loop system. No initial stabilising control is required, and by using recorded and instantaneous data simultaneously for the adaptation of the critic, the restrictive persistence of excitation condition is relaxed. In addition, Lyapunov direct method is utilised to demonstrate the uniform ultimate boundedness of the weights of the observer NN and the critic NN. Finally, an example is provided to verify the effectiveness of the present approach.

Adaptive optimal control for nonlinear discrete-time systems

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems with Control Constraints

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

An Adaptive Cross Approximation-based Method for Robust Nonlinear Feedback Control Problems

Adaptive Neural Inverse Optimal Control for a Class of Strict Feedback Stochastic Nonlinear Systems

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Neural-Network-Based Online Optimal Control for Uncertain Non-Linear Continuous-Time Systems with Control Constraints

Constrained Online Optimal Control for Continuous-Time Nonlinear Systems Using Neuro-Dynamic Programming

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Adaptive Optimal Control for Unknown Constrained Nonlinear Systems with a Novel Quasi-Model Network

Decentralized Adaptive Neural Inverse Optimal Control of Nonlinear Interconnected Systems

Reinforcement Learning for Adaptive Optimal Control of Unknown Continuous-Time Nonlinear Systems with Input Constraints.

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Online Approximate Optimal Control for Affine Non-Linear Systems with Unknown Internal Dynamics Using Adaptive Dynamic Programming

Suboptimal Control of a Class of Nonlinear Singularly Perturbed Systems

Online Optimal Regulation and Tracking Control of Nonlinear Discrete-Time System with Control Constraints

Data-Based On-Line Optimal Control for Unknown Nonlinear Systems Via Adaptive Dynamic Programming Approach

A neural network-based approach for solving quantized discrete-time optimal control with input constraint over finite-horizon.

Adaptive Neural Networks Finite-Time Optimal Control for a Class of Nonlinear Systems