Abstract:Abstract Reinforcement learning (RL) and approximate dynamic programming (ADP) have been recently studied to solve nonlinear optimal control problems (OCPs) of continuous‐time (CT) systems. However, online learning efficiency and reliability are two major concerns to be further improved. Motivated by the above issues, in this paper we propose a receding‐horizon reinforcement learning (RHRL) algorithm for near‐optimal control of CT systems under control constraints. Different from classic RL and ADP, in the proposed approach, the infinite‐horizon OCP is decomposed as a series of finite‐horizon ones solved with an actor‐critic structure according to the receding horizon strategy, which can improve the online learning efficiency and reliability. The unknown dynamics of the system are identified offline using a sparse kernel‐based neural network structure whose weights are also updated online in the RHRL framework to improve the control performance. Moreover, the convergence of the modeling error is proven. To verify the effectiveness of our approach, we apply the RHRL algorithm to the autonomous ground vehicle for realizing near‐optimal path‐tracking control. Compared with CT model predictive control using a nominal model and other model‐free tracking controllers such as pure pursuit, heuristic dual programming, and the soft actor‐critic algorithm, RHRL performs better in terms of control performance.

Receding Horizon Actor–Critic Learning Control for Nonlinear Time-Delay Systems with Unknown Dynamics

A Digital Receding-Horizon Learning Controller for Nonlinear Continuous-time Systems

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Event-triggered Receding Horizon Control Via Actor-Critic Design

Adaptive Control of Nonlinear Time-Varying Processes Using Selective Recursive Kernel Learning Method

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Receding horizon control for a class of nonlinear time-delay systems

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems Using Online Approximators

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems.

Continuous‐time receding‐horizon reinforcement learning and its application to path‐tracking control of autonomous ground vehicles

Learning-based adaptive optimal control of linear time-delay systems: A value iteration approach

Incremental adaptive optimal control for nonlinear systems with disturbance and input time-delay

Non-Predictive Model-Free Control of Nonlinear Systems with Unknown Input Time Delay

Reinforcement Learning-Based Predefined-Time Tracking Control for Nonlinear Systems Under Identifier-Critic-Actor Structure

A Time-Delay Modeling Approach for Data-Driven Predictive Control of Continuous-Time Systems

Reinforcement Learning-Based Adaptive Optimal Control for Nonlinear Systems With Asymmetric Hysteresis

Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

Optimal control of nonlinear system based on deterministic policy gradient with eligibility traces

Optimal dynamic output feedback control of unknown linear continuous-time systems by adaptive dynamic programming

Adaptive Intelligent Control of Nonaffine Nonlinear Time-Delay Systems With Dynamic Uncertainties