Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Connecting Deep-Reinforcement-Learning-based Obstacle Avoidance with Conventional Global Planners using Waypoint Generators

Obstacle-aware Waypoint Generation for Long-range Guidance of Deep-Reinforcement-Learning-based Navigation Approaches

Holistic Deep-Reinforcement-Learning-based Training of Autonomous Navigation Systems

Deep Local Trajectory Replanning and Control for Robot Navigation

Navigation System with SLAM-based Trajectory Topological Map and Reinforcement Learning-Based Local Planner

Deep reinforcement learning-aided autonomous navigation with landmark generators

Enhancing Navigational Safety in Crowded Environments using Semantic-Deep-Reinforcement-Learning-based Navigation

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

All-in-One: A DRL-based Control Switch Combining State-of-the-art Navigation Planners

Off-road Autonomous Vehicles Traversability Analysis and Trajectory Planning Based on Deep Inverse Reinforcement Learning

A Soft Actor-Critic Deep Reinforcement-Learning-Based Robot Navigation Method Using LiDAR

When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning

Integrating Deep Reinforcement Learning with Optimal Trajectory Planner for Automated Driving.

A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance

Adaptive speed planning for Unmanned Vehicle Based on Deep Reinforcement Learning

Learning to Navigate Through Complex Dynamic Environment with Modular Deep Reinforcement Learning

Bridging the Gap: Regularized Reinforcement Learning for Improved Classical Motion Planning with Safety Modules

Learning to Navigate: Exploiting Deep Networks to Inform Sample-Based Planning During Vision-Based Navigation

Efficient Exploration in Crowds by Coupling Navigation Controller and Exploration Planner

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Autonomous Obstacle Avoidance Control Method Based on Safe Adaptive Reinforcement Learning