Abstract:In this paper, we discuss a novel algorithm for learning the solution to the optimal control problem (OCP) for affine nonlinear continuous-time constrained-input systems with completely unknown dynamics on a data-driven integral reinforcement learning (IRL) basis. It is well known that we have to obtain the solution of the nonlinear OCP by means of resolving the Hamilton-Jacobi-Bellman equation (HJBE). However, the HJBE is usually a nonlinear partial differential equation that cannot be solved analytically. To make matters worse, most practical systems are too complex to be accurately mathematically modelled and have real-time errors in the system’s controller. To address the above issues, we propose an online data-driven IRL algorithm that is anchored in policy iteration (PI), using real-time data from practical systems, rather than system models or partially sampled data from systems. To begin with, the PI algorithm is shown. Then, we approximate the performance function and the control policy using a critic neural network (CNN) and an actor neural network (ANN), respectively. The approach presented is an online-policy IRL, where the data are continuously sampled in the input and state domains. The weights of the CNN and ANN are renewed by least squares using the collected data, which minimizes residual errors. Finally, the validity of the approach in solving the OCP is demonstrated from the simulation results.

Synchronous Optimal Control Method for Nonlinear Systems with Saturating Actuators and Unknown Dynamics Using Off-Policy Integral Reinforcement Learning

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Adaptive Optimal Control For A Class Of Uncertain Systems With Saturating Actuators And External Disturbance Using Integral Reinforcement Learning

Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems with Disturbances

Event-triggered-based Online Integral Reinforcement Learning for Optimal Control of Unknown Constrained Nonlinear Systems.

Integral Reinforcement Learning Off-Policy Method for Solving Nonlinear Multi-Player Nonzero-Sum Games with Saturated Actuator.

Online adaptive data-driven control for unknown nonlinear systems with constrained-input

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Online Adaptive Optimal Control Algorithm Based on Synchronous Integral Reinforcement Learning With Explorations

Online Synchronous Iterative Algorithm for Optimal Control of Stochastic Affine Nonlinear Systems

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Neural-Network-Based Online Optimal Control for Uncertain Non-Linear Continuous-Time Systems with Control Constraints

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Neuro-Optimal Control of Unknown Nonaffine Nonlinear Systems with Saturating Actuators.

Optimal Robust Control of Nonlinear Uncertain System Via Off-Policy Integral Reinforcement Learning

Online adaptive learning of optimal control solutions using integral reinforcement learning

Optimal Synchronization Control of Multiagent Systems with Input Saturation Via Off-Policy Reinforcement Learning.

Optimal Control Laws For Nonlinear Oscillator Systems With Saturating Actuators Using Neural Networks Based On Policy Iteration

Off-policy Neuro-Optimal Control for Unknown Complex-Valued Nonlinear Systems Based on Policy Iteration

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Nearly Optimal Control for Mixed Zero-Sum Game Based on Off-Policy Integral Reinforcement Learning