Abstract:Abstract Reinforcement learning (RL) and approximate dynamic programming (ADP) have been recently studied to solve nonlinear optimal control problems (OCPs) of continuous‐time (CT) systems. However, online learning efficiency and reliability are two major concerns to be further improved. Motivated by the above issues, in this paper we propose a receding‐horizon reinforcement learning (RHRL) algorithm for near‐optimal control of CT systems under control constraints. Different from classic RL and ADP, in the proposed approach, the infinite‐horizon OCP is decomposed as a series of finite‐horizon ones solved with an actor‐critic structure according to the receding horizon strategy, which can improve the online learning efficiency and reliability. The unknown dynamics of the system are identified offline using a sparse kernel‐based neural network structure whose weights are also updated online in the RHRL framework to improve the control performance. Moreover, the convergence of the modeling error is proven. To verify the effectiveness of our approach, we apply the RHRL algorithm to the autonomous ground vehicle for realizing near‐optimal path‐tracking control. Compared with CT model predictive control using a nominal model and other model‐free tracking controllers such as pure pursuit, heuristic dual programming, and the soft actor‐critic algorithm, RHRL performs better in terms of control performance.

Online Optimal Control of Robotic Systems with Single Critic NN-Based Reinforcement Learning.

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Near Optimal Neural Network-based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

Adaptive Identifier-Critic-Based Optimal Tracking Control for Nonlinear Systems with Experimental Validation

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Robust Neurooptimal Control for a Robot Via Adaptive Dynamic Programming

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Model-Based Actor-Critic Learning for Optimal Tracking Control of Robots with Input Saturation.

Tracking Control Optimization Scheme of Continuous-Time Nonlinear System Via Online Single Network Adaptive Critic Design Method.

Model-free Nonlinear Robust Control Design Via Online Critic Learning.

Online Actor-Critic Learning for Motion Control of Non-holonomic Mobile Robot

Neural-network-based Reinforcement Learning Controller for Nonlinear Systems with Non-Symmetric Dead-Zone Inputs

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Neural‐based Online Finite‐time Optimal Tracking Control for Wheeled Mobile Robotic System with Inequality Constraints

Online Adaptive Optimal Tracking Control of Nonholonomic Mobile Robot

Adaptive Neural Network Control of Robot Manipulator Using Reinforcement Learning

Neural-Network-Based Online Optimal Control for Uncertain Non-Linear Continuous-Time Systems with Control Constraints

Continuous‐time receding‐horizon reinforcement learning and its application to path‐tracking control of autonomous ground vehicles

Robust Tracking Control of Uncertain Nonlinear Systems with Adaptive Dynamic Programming

Online Optimal Regulation and Tracking Control of Nonlinear Discrete-Time System with Control Constraints