Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

RHSOA: Receding Horizon Optimization-Based Swift Obstacle Avoidance Approach for UAVs in Unknown Environments

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Reactive Obstacle Avoidance Method for a UAV

An Avoiding Obstacles Method Of Multi-Rotor Uavs

Obstacle avoidance for autonomous Ground Vehicles based on moving horizon optimization

Reactive Obstacle Avoidance for Unmanned Helicopter Using Fuzzy Control

Monocular Obstacle Avoidance Based on Inverse PPO for Fixed-wing UAVs

Integrated Real Time Obstacle Avoidance Algorithm Based on Fuzzy Logic and L1 Control Algorithm for Unmanned Helicopter

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

Obstacle Avoidance Methods for Rotor UAVs Using RealSense Camera

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

A Real-time Obstacle Avoidance Algorithm for UAV in 2D Space

Receding horizon control for multiple UAV formation flight based on modified brain storm optimization

Reactive Navigation of an Unmanned Aerial Vehicle with Perception-based Obstacle Avoidance Constraints

Robust Vision-based Obstacle Avoidance for Micro Aerial Vehicles in Dynamic Environments

Collision Avoidance for Cooperative Uavs with Rolling Optimization Algorithm Based on Predictive State Space

Reducing Oscillations for Obstacle Avoidance in a Dense Environment Using Deep Reinforcement Learning and Time-Derivative of an Artificial Potential Field

Learning-based fixed-wing UAV reactive maneuver control for obstacle avoidance

Chaotic Biogeography-Based Optimization Approach to Receding Horizon Control for Multiple UAVs Formation Flight

Perception-Aware Baesd Quadrotor Translation and Yaw Angle Planner in Unpredictable Dynamic Situations

A Saliency-Based Reinforcement Learning Approach for a UAV to Avoid Flying Obstacles