Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Trajectory Tracking and Optimal Obstacle Avoidance of Mobile Agent Based on Data-Driven Control

Robust Hierarchical Model Predictive Control for Trajectory Tracking with Obstacle Avoidance

Modeling and Optimal Control of Omni-Directional Mobile Robots

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

A Novel Virtual-Structure Formation Control Design for Mobile Robots with Obstacle Avoidance

Autonomous Obstacle Avoidance and Trajectory Planning for Mobile Robot Based on Dual-Loop Trajectory Tracking Control and Improved Artificial Potential Field Method

A Multi-Switching Tracking Control Scheme for Autonomous Mobile Robot in Unknown Obstacle Environments

Unpredictable Trajectory Design for Mobile Agents.

A Path Planning and Tracking Control for Autonomous Vehicle With Obstacle Avoidance

Data-Driven Trajectory Tracking Control Algorithm Design for Fast Migration to Different Autonomous Vehicles

Model Predictive Trajectory Optimization and Tracking in Highly Constrained Environments

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Motion Control And Trajectory Planning For Obstacle Avoidance Of The Mobile Parallel Robot Driven By Three Tracked Vehicles

Adaptive Tracking And Obstacle Avoidance Control For Mobile Robots With Unknown Sliding

Measurement-Based method for nonholonomic mobile vehicles with obstacle avoidance

An adaptive trajectory tracking control of wheeled mobile robots

Trajectory Planning And Tracking Control Of Vehicle Obstacle Avoidance Based On Optimization Control

Trajectory tracking control for underactuated autonomous vehicles via adaptive dynamic programming

Obstacle-circumventing adaptive control of a four-wheeled mobile robot subjected to motion uncertainties

Leader-Following Formation Tracking Control of Nonholonomic Mobile Robots Considering Collision Avoidance: A System Transformation Approach

Constraint‐Oriented Obstacle Avoidance Control for Autonomous Vehicles Without Local Trajectory Replanning