Abstract:Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the long-term execution. An alternative method is to train a DRL policy that outputs the navigation path directly. Then the robot can follow the generated path smoothly using sophisticated velocity-planning and path-following controllers, whose parameters are specified according to the hardware platform. However, two roadblocks arise for training a DRL policy that outputs paths:(1) The action space for potential paths often involves higher dimensions comparing to low-level commands, which increases the difficulties of training; (2) It takes multiple time steps to track a path instead of a single time step, which requires the path to predicate the interactions of the robot w.r.t. the dynamic environment in multiple time steps. This, in turn, amplifies the challenges associated with training. In response to these challenges, we propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. Specifically, we employ specific action space discretization techniques and tailored state space representation methods to address the associated challenges. Curriculum learning is employed to expedite the training process, while the reward function also takes into account the smooth transition between adjacent paths. In our experiments, PathRL achieves better success rates and reduces angular rotation variability compared to other DRL navigation methods, facilitating stable and smooth robot movement. We demonstrate the competitive edge of PathRL in both real-world scenarios and multiple challenging simulation environments.

An Experience Aggregative Reinforcement Learning With Multi-Attribute Decision-Making for Obstacle Avoidance of Wheeled Mobile Robot

Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot

The Design and Realization of Multi-agent Obstacle Avoidance based on Reinforcement Learning

Multi-objective deep reinforcement learning for crowd-aware robot navigation with dynamic human preference

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

Autonomous Obstacle Avoidance Control Method Based on Safe Adaptive Reinforcement Learning

Adaptive Aggregation for Safety-Critical Control

Multi-Uav Automatic Dynamic Obstacle Avoidance With Experience-Shared A2c

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Reinforcement Learning For Mobile Robot Obstacle Avoidance Under Dynamic Environments

Reinforcement Learned Distributed Multi-Robot Navigation With Reciprocal Velocity Obstacle Shaped Rewards

Improved M-DQN with $\Epsilon$-Ucb Action Selection Policy and Multi-Goal Fusion Reward Function for Mobile Robot Path Planning

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

Enhancing Robotic Navigation: An Evaluation of Single and Multi-Objective Reinforcement Learning Strategies

Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning

Cooperative Control of Multiple AGVs Based on Multi-Agent Reinforcement Learning

Dynamic Obstacle Avoidance Technique for Mobile Robot Navigation Using Deep Reinforcement Learning

End-to-End Autonomous Navigation Based on Deep Reinforcement Learning with a Survival Penalty Function

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Multi-Agent Reinforcement Learning-Based UAV Pathfinding for Obstacle Avoidance in Stochastic Environment

PathRL: an End-To-End Path Generation Method for Collision Avoidance Via Deep Reinforcement Learning