Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Learning-Based Safe Control for Robot and Autonomous Vehicle Using Efficient Safety Certificate

Synthesize Efficient Safety Certificates for Learning-Based Safe Control Using Magnitude Regularization

Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

Joint Synthesis of Safety Certificate and Safe Control Policy Using Constrained Reinforcement Learning

Safe Control With Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction Methods for Robotics and Control

Safe Multiagent Learning with Soft Constrained Policy Optimization in Real Robot Control

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

Model-Free Safe Reinforcement Learning Through Neural Barrier Certificate

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Learning Safe Control for Multi-Robot Systems: Methods, Verification, and Open Challenges

Safety-Enhanced Self-Learning for Optimal Power Converter Control

Safe Reinforcement Learning Algorithm and Its Application in Intelligent Control for CPS

Safe Reinforcement Learning-Based Eco-Driving Control for Mixed Traffic Flows With Disturbances

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Energy-Efficient Control Adaptation with Safety Guarantees for Learning-Enabled Cyber-Physical Systems

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

Adaptive Safe Reinforcement Learning with Full-State Constraints and Constrained Adaptation for Autonomous Vehicles

A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

Safe Online Dynamics Learning with Initially Unknown Models and Infeasible Safety Certificates

Safety-Aware Preference-Based Learning for Safety-Critical Control