Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

An Efficient and Responsive Robot Motion Controller for Safe Human-Robot Collaboration

An Autonomous Robot for Collision-Free Person Following Through Model Predictive Control

Efficient Motion Planning Based on Kinodynamic Model for Quadruped Robots Following Persons in Confined Spaces

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

Optimal Motion Planning under Dynamic Risk Region for Safe Human–Robot Cooperation

Human Intention-Aware Motion Planning and Adaptive Fuzzy Control for a Collaborative Robot With Flexible Joints

Adaptive Safety-critical Control with Uncertainty Estimation for Human-robot Collaboration

Reinforcement Learning Based User-Guided Motion Planning for Human-Robot Collaboration

Motion Planning Combines Psychological Safety and Motion Prediction for a Sense Motive Robot

A Real-Time Hierarchical Control Method for Safe Human–robot Coexistence

Online Collision Avoidance for Human-Robot Collaborative Interaction Concerning Safety and Efficiency

Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction

A Dynamic Planner for Safe and Predictable Human-Robot Collaboration

Safety Guaranteed Manipulation Based on Reinforcement Learning Planner and Model Predictive Control Actor

Adaptive Admittance Control for Safety-Critical Physical Human Robot Collaboration

Motion Planning Combines Human Motion Prediction for Human-Robot Cooperation

Hybrid Trajectory Replanning-Based Dynamic Obstacle Avoidance for Physical Human-Robot Interaction

Relevance-driven Decision Making for Safer and More Efficient Human Robot Collaboration

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Energy-Efficient Motion Planning and Control for Robotic Arms Via Deep Reinforcement Learning

An Active Strategy for Safe Human–Robot Interaction Based on Visual–Tactile Perception