Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Adaptive Control and Intersections with Reinforcement Learning

Adaptive Robust Control Of Nonlinear Systems With Dynamic Uncertainties

Dual Adaptive Control

Decentralized Adaptive Iterative Learning Control for Interconnected Systems with Uncertainties

Training for More Robust and Practical Adaptive Signal Control Models

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Adaptive robust control algorithm for enhanced path-tracking performance of automated driving in critical scenarios

Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equations

Adaptive control of a mechatronic system using constrained residual reinforcement learning

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

Robust Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Both Parametric and Nonparametric Uncertainties

Scalable Robust Adaptive Control from the System Level Perspective

Adaptive Event-triggered Reinforcement Learning Control for Complex Nonlinear Systems

Deep Adaptive Control: Deep Reinforcement Learning-Based Adaptive Vehicle Trajectory Control Algorithms for Different Risk Levels

Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control

A Harmonized Approach: Beyond-the-Limit Control for Autonomous Vehicles Balancing Performance and Safety in Unpredictable Environments

A Recurrent Reinforcement Learning Approach Applicable To Highly Uncertain Environments

Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control