Abstract:The state and input constraints of nonlinear systems could greatly impede the realization of their optimal control when using reinforcement learning (RL)-based approaches since the commonly used quadratic utility functions cannot meet the requirements of solving constrained optimization problems. This article develops a novel optimal control approach for constrained discrete-time (DT) nonlinear systems based on safe RL. Specifically, a barrier function (BF) is introduced and incorporated with the value function to help transform a constrained optimization problem into an unconstrained one. Meanwhile, the minimum of such an optimization problem can be guaranteed to occur at the origin. Then a constrained policy iteration (PI) algorithm is developed to realize the optimal control of the nonlinear system and to enable the state and input constraints to be satisfied. The constrained optimal control policy and its corresponding value function are derived through the implementation of two neural networks (NNs). Performance analysis shows that the proposed control approach still retains the convergence and optimality properties of the traditional PI algorithm. Simulation results of three examples reveal its effectiveness.

Reinforcement learning for optimal control of linear impulsive systems with periodic impulses

Reinforcement Learning-Based $\mathcal{h}_{\infty }$ Control of 2-D Markov Jump Roesser Systems with Optimal Disturbance Attenuation

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Optimal control of impulsive Volterra equations with variable impulse times

Impulsive Control of Nonlinear Systems with Time-Varying Delays

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Reinforcement Learning for Finite-Horizon H∞ Tracking Control of Unknown Discrete Linear Time-Varying System

Event‐based output feedback control of periodic systems: A piecewise impulsive method

Stabilization on periodic impulse control systems

A novel finite-time stability criteria and controller design for nonlinear impulsive systems

Learning-based adaptive optimal control of linear time-delay systems: A value iteration approach

Hybrid Reinforcement Learning for Optimal Control of Non-Linear Switching System

Impulsive modeling and control of a new chaotic system

Near-Optimal Control for Time-Varying Linear Discrete Systems with Additive Nonlinearities and Random Gains

Finite-time stabilization of nonlinear time-varying systems involving impulsive action

Stability and Stabilization of Linear Impulsive Systems with Large Impulse-Delays: A Stabilizing Delay Perspective

Data-Driven Near-Optimal Control of Nonlinear Systems Over Finite Horizon

Physics‐informed reinforcement learning for optimal control of nonlinear systems

NN Reinforcement Learning Adaptive Control for a Class of Nonstrict-Feedback Discrete-Time Systems

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme