Abstract:Home service robots prioritize cost-effectiveness and convenience over the precision required for industrial tasks like autonomous driving, making their task execution more easily. Meanwhile, path planning tasks using Deep Reinforcement Learning(DRL) are commonly sparse reward problems with limited data utilization, posing challenges in obtaining meaningful rewards during training, consequently resulting in slow or challenging training. In response to these challenges, our paper introduces a lightweight end-to-end path planning algorithm employing with hindsight experience replay(HER). Initially, we optimize the reinforcement learning training process from scratch and map the complex high-dimensional action space and state space to the representative low-dimensional action space. At the same time, we improve the network structure to decouple the model navigation and obstacle avoidance module to meet the requirements of lightweight. Subsequently, we integrate HER and curriculum learning (CL) to tackle issues related to inefficient training. Additionally, we propose a multi-step hindsight experience replay (MS-HER) specifically for the path planning task, markedly enhancing both training efficiency and model generalization across diverse environments. To substantiate the enhanced training efficiency of the refined algorithm, we conducted tests within diverse Gazebo simulation environments. Results of the experiments reveal noteworthy enhancements in critical metrics, including success rate and training efficiency. To further ascertain the enhanced algorithm's generalization capability, we evaluate its performance in some "never-before-seen" simulation environment. Ultimately, we deploy the trained model onto a real lightweight robot for validation. The experimental outcomes indicate the model's competence in successfully executing the path planning task, even on a small robot with constrained computational resources.

Hindsight Planner.

Exploration via Hindsight Goal Generation

Combining Hindsight with Goal-enhanced Prediction for Multi-goal Reinforcement Learning

Improvements on Hindsight Learning

Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency

Efficient Object Manipulation to an Arbitrary Goal Pose: Learning-based Anytime Prioritized Planning

Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learning

Reinforcement learning path planning method incorporating multi-step Hindsight Experience Replay for lightweight robots

Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning

Addressing Hindsight Bias in Multigoal Reinforcement Learning

MHER: Model-based Hindsight Experience Replay

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

Soft Hindsight Experience Replay

Quantile Regression Hindsight Experience Replay

Leveraging Efficiency Through Hybrid Prioritized Experience Replay in Door Environment.

Explicit-Implicit Subgoal Planning for Long-Horizon Tasks with Sparse Reward

MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards

ACDER: Augmented Curiosity-Driven Experience Replay

Cluster-based Sampling in Hindsight Experience Replay for Robotic Tasks (Student Abstract)

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning