Abstract:In this paper, we discuss a novel algorithm for learning the solution to the optimal control problem (OCP) for affine nonlinear continuous-time constrained-input systems with completely unknown dynamics on a data-driven integral reinforcement learning (IRL) basis. It is well known that we have to obtain the solution of the nonlinear OCP by means of resolving the Hamilton-Jacobi-Bellman equation (HJBE). However, the HJBE is usually a nonlinear partial differential equation that cannot be solved analytically. To make matters worse, most practical systems are too complex to be accurately mathematically modelled and have real-time errors in the system’s controller. To address the above issues, we propose an online data-driven IRL algorithm that is anchored in policy iteration (PI), using real-time data from practical systems, rather than system models or partially sampled data from systems. To begin with, the PI algorithm is shown. Then, we approximate the performance function and the control policy using a critic neural network (CNN) and an actor neural network (ANN), respectively. The approach presented is an online-policy IRL, where the data are continuously sampled in the input and state domains. The weights of the CNN and ANN are renewed by least squares using the collected data, which minimizes residual errors. Finally, the validity of the approach in solving the OCP is demonstrated from the simulation results.

Adaptive Optimal Control for a Class of Continuous-Time Affine Nonlinear Systems with Unknown Internal Dynamics

Adaptive optimal control for a class of nonlinear partially uncertain dynamic systems via policy iteration

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Online Approximate Optimal Control for Affine Non-Linear Systems with Unknown Internal Dynamics Using Adaptive Dynamic Programming

Neural-Network-Based Online Optimal Control for Uncertain Non-Linear Continuous-Time Systems with Control Constraints

Reinforcement Learning for Adaptive Optimal Control of Unknown Continuous-Time Nonlinear Systems with Input Constraints.

Adaptive Optimal Control for a Class of Nonlinear Systems: the Online Policy Iteration Approach.

Neuro-Optimal Control of Unknown Nonaffine Nonlinear Systems with Saturating Actuators.

Constrained Online Optimal Control for Continuous-Time Nonlinear Systems Using Neuro-Dynamic Programming

Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems

Adaptive Optimal Control for Unknown Constrained Nonlinear Systems with a Novel Quasi-Model Network

Online adaptive learning of optimal control solutions using integral reinforcement learning

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Finite-horizon Optimal Control of Unknown Nonlinear Time-Delay Systems

Online Adaptive Policy Learning Algorithm for H-Infinity State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems

Online adaptive data-driven control for unknown nonlinear systems with constrained-input

Adaptive optimal control for nonlinear discrete-time systems

Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems