Abstract:Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. For such safety-critical systems, it will certainly be a requirement that safe performance should be ensured even during the reinforcement learning period in the presence of system uncertainty. To address this issue, a Barrier Lyapunov Function-based safe reinforcement learning algorithm (BLF-SRL) is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges the Barrier Lyapunov Function item into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning when unknown bounded system uncertainty exists. More specifically, the overall system control is optimized with the optimized backstepping technique under the framework of Actor-Critic, which optimizes the virtual control in every backstepping subsystem. Wherein, the optimal virtual control is decomposed into Barrier Lyapunov Function items; and also with an adaptive item to be learned with deep neural networks, which achieves safe exploration during the learning process. Eventually, the principle of Bellman optimality is satisfied through iteratively updating the independently approximated actor and critic to solve the Hamilton-Jacobi-Bellman equation in adaptive dynamic programming. More notably, the variance of control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with motion control problems for autonomous vehicles through appropriate comparison simulations.

Research on Solving Nonlinear Problem of Ball and Beam System by Introducing Detail-Reward Function

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Enhanced stability dynamic surface control for aircraft antiskid braking system using electromechanical actuator

Constrained Dynamic Surface Control of Electro-Hydrostatic Actuator Anti-Skid Braking System

Learning Ball-balancing Robot Through Deep Reinforcement Learning

A Balance Control Method for Wheeled Bipedal Robot Based on Reinforcement Learning

Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles with Optimized Backstepping

Design of Ball-Beam Control System Based on Machine Vision

Research of Ball and Beam System's Control Method

A Novel Adaptive Control Algorithm Based On Reinforcement Learning

Evolving a Neural Controller for a Ball-and-beam System

Reinforcement Learning-Based Control of Nonlinear Systems Using Lyapunov Stability Concept and Fuzzy Reward Scheme

Parameter Refinement of a Ballbot and Predictive Control for Reference Tracking with Linear Parameter-Varying Embedding

Aircraft Anti-Skid Braking Control Based on Reinforcement Learning

Adaptive Neural Network Optimal Backstepping Control of Strict Feedback Nonlinear Systems Via Reinforcement Learning

Dynamic Obstacle Avoidance Using Nonlinear Model Predictive Control and Control Barrier Function for Ballbot Systems

RBF Neural Network Based Dynamic Surface Control for Anti-Skid Braking System Using Barrier Lyapunov Theorem

On Stability for Sampled-Data Nonlinear ADRC-based Control System with Application to the Ball-Beam Problem.

Balance Controller Design for Inverted Pendulum Considering Detail Reward Function and Two-Phase Learning Protocol