Abstract:Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the long-term execution. An alternative method is to train a DRL policy that outputs the navigation path directly. Then the robot can follow the generated path smoothly using sophisticated velocity-planning and path-following controllers, whose parameters are specified according to the hardware platform. However, two roadblocks arise for training a DRL policy that outputs paths:(1) The action space for potential paths often involves higher dimensions comparing to low-level commands, which increases the difficulties of training; (2) It takes multiple time steps to track a path instead of a single time step, which requires the path to predicate the interactions of the robot w.r.t. the dynamic environment in multiple time steps. This, in turn, amplifies the challenges associated with training. In response to these challenges, we propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. Specifically, we employ specific action space discretization techniques and tailored state space representation methods to address the associated challenges. Curriculum learning is employed to expedite the training process, while the reward function also takes into account the smooth transition between adjacent paths. In our experiments, PathRL achieves better success rates and reduces angular rotation variability compared to other DRL navigation methods, facilitating stable and smooth robot movement. We demonstrate the competitive edge of PathRL in both real-world scenarios and multiple challenging simulation environments.

An Incremental Optimization Approach to Address the Spatiotemporal Reward Coupling Effects in Deep Reinforcement Learning for Path Planning

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

Robot Path Planning in Dynamic Environments Based on Deep Reinforcement Learning

An End-to-End Path Planner Combining Potential Field Method With Deep Reinforcement Learning

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

An Improved Deep Reinforcement Learning Algorithm for Path Planning in Unmanned Driving

A Path Planning Method Based on Deep Reinforcement Learning with Improved Prioritized Experience Replay for Human-Robot Collaboration

Multiple Suboptimal Policies Integrated Reinforcement Learning Algorithm for Path Planning

Deep Reinforcement Learning Based Path Planning for Mobile Robots Using Time-Sensitive Reward

Dynamic Path Planning for Mobile Robots with Deep Reinforcement Learning

A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance

Deep Reinforcement Learning for Indoor Mobile Robot Path Planning

PathRL: an End-To-End Path Generation Method for Collision Avoidance Via Deep Reinforcement Learning

A Mapless Local Path Planning Approach Using Deep Reinforcement Learning Framework

Path Planning of Autonomous Mobile Robot in Comprehensive Unknown Environment Using Deep Reinforcement Learning

Research on Game-Playing Agents Based on Deep Reinforcement Learning.

An Efficiently Convergent Deep Reinforcement Learning-Based Trajectory Planning Method for Manipulators in Dynamic Environments

Research on Mobile Robot Path Planning Based on Deep Reinforcement Learning

A decentralized path planning model based on deep reinforcement learning

AGV Path Planning Using Curiosity-Driven Deep Reinforcement Learning