Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

An Improved Sequential Convex Programming Obstacle Avoidance Algorithm for Autonomous Vehicles

Improved Sequential Convex Programming Based on Pseudospectral Discretization for Entry Trajectory Optimization

Efficient Optimization-Based Trajectory Planning for Unmanned Systems in Confined Environments

Exact Obstacle Avoidance for Autonomous Vehicles in Polygonal Domains

Trajectory Optimization with Complex Obstacle Avoidance Constraints Via Homotopy Network Sequential Convex Programming

An Incremental Strategy for Tractor-Trailer Vehicle Global Trajectory Optimization in the Presence of Obstacles.

Optimal Vehicle Trajectory Planning for Static Obstacle Avoidance using Nonlinear Optimization

Real-Time Optimal Trajectory Planning for Autonomous Driving with Collision Avoidance Using Convex Optimization

Online trajectory optimization for safe autonomous overtaking with active obstacle avoidance

An Efficient Spatial-Temporal Trajectory Planner for Autonomous Vehicles in Unstructured Environments

Efficient collision avoidance for autonomous vehicles in polygonal domains

Hierarchical collision-free trajectory planning for autonomous vehicles based on improved artificial potential field method

Enhancing Planning for Autonomous Driving via an Iterative Optimization Framework Incorporating Safety-Critical Trajectory Generation

TDR-OBCA: A Reliable Planner for Autonomous Driving in Free-Space Environment

Efficient optimization-based trajectory planning

Obstacle Avoidance Path Planning Design for Autonomous Driving Vehicles Based on an Improved Artificial Potential Field Algorithm

Spatio-Temporal Trajectory Planning Using Search And Optimizing Method for Autonomous Driving

Motion planning of multiple unmanned vehicles using sequential convex programming‐based distributed model predictive control

Path Planning and Obstacle Avoidance Control for Autonomous Multi-Axis Distributed Vehicle Based on Dynamic Constraints

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem