Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Neuro-endocrine Computation Model and Its Application in Obstacle Avoidance

A Robot Collision Avoidance Scheme Based on the Moving Obstacle Motion Prediction

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Intelligent Obstacle Avoidance Algorithm for Mobile Robots in Uncertain Environment

A Dynamic Control Model for Modulating Using Artificial Neural Networks Using the Artificial Endocrine System

Random Neural Q-Learning for Obstacle Avoidance of A Mobile Robot in Unknown Environments

A Neuro-Cognitive System and Its Application in Robotics

Firing Feature-Driven Neural Circuits with Scalable Memristive Neurons for Robotic Obstacle Avoidance

A Neuromorphic Approach to Obstacle Avoidance in Robot Manipulation

A Neural Networks Algorithm for Robot Path Planning in Unknown Environment

Integrated model of cerebellal supervised learning and basal ganglia's reinforcement learning for mobile robot behavioral decision-making

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle Avoidance

Nature-inspired self-organizing collision avoidance for drone swarm based on reward-modulated spiking neural network

Learning Obstacle Avoidance Behavior Using Multi-Agent Learning With Fuzzy States

An Adaptive Teleoperate Mobile Robot Controller Based on Fuzzy Neural Network

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

Adaptive teleoperated mobile robot controller based on fuzzy neural network

Research on Cooperative Obstacle Avoidance Decision Making of Unmanned Aerial Vehicle Swarms in Complex Environments under End-Edge-Cloud Collaboration Model

Vision attention learning model and its application in robot

Ultrasonic Sensor Based Fuzzy-Neural Control Algorithm of Obstacle Avoidance for Mobile Robot