Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

The podocin mutation R229Q and early recurrence (within the first year) of glomerular disease after renal transplantation.

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

A Haptic Shared Control Architecture for Tracking of a Moving Object.

Adaptive Environment-Aware Robotic Arm Reaching Based on a Bio-Inspired Neurodynamical Computational Framework

Unified Motion/Force/Impedance Control for Manipulators in Unknown Contact Environments Based on Robust Model-Reaching Approach

Kinematic-Model-Free Predictive Control for Robotic Manipulator Target Reaching With Obstacle Avoidance

Predictive Multi-Agent-Based Planning and Landing Controller for Reactive Dual-Arm Manipulation

Fuzzy Adaptive Control-based Real-time Obstacle Avoidance under Uncertain Perturbations

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

SERA: Safe and Efficient Reactive Obstacle Avoidance for Collaborative Robotic Planning in Unstructured Environments

Hierarchical Control of Visually-Guided Movements in a 3D-Printed Robot Arm

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

Learning robotic eye–arm–hand coordination from human demonstration: a coupled dynamical systems approach

An obstacle avoidance method for robotic arm based on reinforcement learning

Unified Control Framework for Real-Time Interception and Obstacle Avoidance of Fast-Moving Objects with Diffusion Variational Autoencoder

Reach then See: A New Adaptive Controller for Robot Manipulator Based on Dual Task-Space Information

Obstacle- and Occlusion-Responsive Visual Tracking Control for Redundant Manipulators using Reachability Measure

A Novel Planning and Tracking Approach for Mobile Robotic Arm in Obstacle Environment

Robot target tracking control considering obstacle avoidance based on combination of deep reinforcement learning and PID

Human Intention-Aware Motion Planning and Adaptive Fuzzy Control for a Collaborative Robot With Flexible Joints

Real-Time Kinodynamic Motion Planning for Omnidirectional Mobile Robot Soccer using Rapidly-Exploring Random Tree in Dynamic Environment with Moving Obstacles