Abstract:This paper presents a novel composite obstacle avoidance control method to generate safe motion trajectories for autonomous systems in an adaptive manner. First, system safety is described using forward invariance, and the barrier function is encoded into the cost function such that the obstacle avoidance problem can be characterized by an infinite-horizon optimal control problem. Next, a safe reinforcement learning framework is proposed by combining model-based policy iteration and state-following-based approximation. Upon real-time data and extrapolated experience data, this learning design is implemented through the actor-critic structure, in which critic networks are tuned by gradient-descent adaption and actor networks produce adaptive control policies via gradient projection. Then, system stability and weight convergence are theoretically analyzed using Lyapunov method. Finally, the proposed learning-based controller is demonstrated on a two-dimensional single integrator system and a nonlinear unicycle kinematic system. Simulation results reveal that the system or agent can smoothly reach the target point while keeping a safe distance from each obstacle; at the same time, other three avoidance control methods are used to provide side-by-side comparisons and to verify some claimed advantages of the present method. Note to Practitioners—This paper is motivated by the obstacle avoidance problem of real-time navigation of an agent to the target point, which applies to practical autonomous systems such as vehicles and robots. Pre-generative methods and reactive methods have been widely employed to generate safe motion trajectories in the obstacle environment. However, these methods cannot strike a good balance between safety and optimality. In this paper, the obstacle avoidance problem is formulated in the sense of optimal control, and a safe reinforcement learning method is designed to generate safe motion trajectories. This method combines the advantages of model-based policy iteration and state-following-based approximation, in which the former ensures regional optimality while the latter ensures local safety. Based on the proposed adaptive tuning laws, engineers are able to design learning-based avoidance controllers in the environment with static obstacles. In future research, we will address the dynamic avoidance problem against moving obstacles.

Dynamic interpolation for obstacle avoidance on Riemannian manifolds

Study of dynamical system based obstacle avoidance via manipulating orthogonal coordinates

On the Design of Region-Avoiding Metrics for Collision-Safe Motion Generation on Riemannian Manifolds

Accelerated Gradient Dynamics on Riemannian Manifolds: Faster Rate and Trajectory Convergence

Reduction of Sufficient Conditions in Variational Obstacle Avoidance Problems

Piecewise-Bézier C1 Interpolation on Riemannian Manifolds with Application to 2D Shape Morphing

C1 interpolating Bézier path on Riemannian manifolds, with applications to 3D shape space

Collision Avoidance using Iterative Dynamic and Nonlinear Programming with Adaptive Grid Refinements

A Novel Approach to Coupling Terms to Avoid Obstacles in a Manipulator Movement Reproduction

Guaranteed Reachability on Riemannian Manifolds for Unknown Nonlinear Systems

A Simple Dynamic Model for Aggressive, Near-Limits Trajectory Planning

Safe Reinforcement Learning and Adaptive Optimal Control With Applications to Obstacle Avoidance Problem

Trajectory Optimization and Force Control with Modified Dynamic Movement Primitives under Curved Surface Constraints.

Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environments

Obstacle Avoidance for Kinematically Redundant Manipulators Using Polyhedral Approximations

Optimization on manifolds: A symplectic approach

Multivariable Analytic Interpolation with Complexity Constraints: A Modified Riccati Approach

Closed-Form Minkowski Sum Approximations for Efficient Optimization-Based Collision Avoidance

Fitting Smoothing Splines to Time-Indexed, Noisy Points on Nonlinear Manifolds

An Intent Inference Based Dynamic Obstacle Avoidance Method For Intelligent Vehicle In Structured Environment

Nonsmooth Control Barrier Functions for Obstacle Avoidance between Convex Regions