Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Reactive Self-rescue Control for Autonomous Mobile Robot Based on Reinforcement Learning

RL and ANN Based Modular Path Planning Controller for Resource-Constrained Robots in the Indoor Complex Dynamic Environment

Autonomous Obstacle Avoidance Control Method Based on Safe Adaptive Reinforcement Learning

Navigation for Mobile Robots Using Reinforcement Learning and Fuzzy Logic

A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots

Reinforcement Learning For Mobile Robot Obstacle Avoidance Under Dynamic Environments

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

Real-time self-reaction of mobile robot with genetic fuzzy neural network in unknown environment

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Mobile Robot Navigation Control Based on Hierarchical Reinforcement Learning

Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

Intelligent mobile robot navigation in unknown and complex environment using reinforcement learning technique

Reinforcement Learning-Based Path Planning Algorithm for Mobile Robots

Autonomous Learning and Navigation of Mobile Robots Based on Deep Reinforcement Learning

Autonomous reinforcement learning control for space robot to capture non-cooperative targets

Research on Motion Planning Based on Flocking Control and Reinforcement Learning for Multi-Robot Systems

Learning to Recover for Safe Reinforcement Learning

Robot Navigation with Reinforcement Learned Path Generation and Fine-Tuned Motion Control

Reset-Free Reinforcement Learning via Multi-State Recovery and Failure Prevention for Autonomous Robots

Robot obstacle avoidance system using deep reinforcement learning