Abstract:Robotic motion planning in dense and dynamic indoor scenarios constantly challenges the researchers because of the motion unpredictability of obstacles. Recent progress in reinforcement learning enables robots to better cope with the dense and unpredictable obstacles by encoding complex features of the robot and obstacles into the encoders like the long-short term memory (LSTM). Then these features are learned by the robot using reinforcement learning algorithms, such as the deep Q network and asynchronous advantage actor critic algorithm. However, existing methods depend heavily on expert experiences to enhance the convergence speed of the networks by initializing them via imitation learning. Moreover, those approaches based on LSTM to encode the obstacle features are not always efficient and robust enough, therefore sometimes causing the network overfitting in training. This paper focuses on the advantage actor critic algorithm and introduces an attention-based actor critic algorithm with experience replay algorithm to improve the performance of existing algorithm from two perspectives. First, LSTM encoder is replaced by a robust encoder attention weight to better interpret the complex features of the robot and obstacles. Second, the robot learns from its past prioritized experiences to initialize the networks of the advantage actor-critic algorithm. This is achieved by applying the prioritized experience replay method, which makes the best of past useful experiences to improve the convergence speed. As results, the network based on our algorithm takes only around 15% and 30% experiences to get rid of the early-stage training without the expert experiences in cases with five and ten obstacles, respectively. Then it converges faster to a better reward with less experiences (near 45% and 65% of experiences in cases with ten and five obstacles respectively) when comparing with the baseline LSTM-based advantage actor critic algorithm. Our source code is freely available at the GitHub (https://github.com/CHUENGMINCHOU/AW-PER-A2C).

An advantage actor-critic algorithm for robotic motion planning in dense and dynamic scenarios

Attention-based advantage actor-critic algorithm with prioritized experience replay for complex 2-D robotic motion planning

A review of motion planning algorithms for intelligent robotics

Hybrid of representation learning and reinforcement learning for dynamic and complex robotic motion planning

Dynamic Path Planning for Mobile Robots with Deep Reinforcement Learning

Representation Learning and Reinforcement Learning for Dynamic Complex Motion Planning System

A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots

A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance

Energy-Efficient Motion Planning and Control for Robotic Arms Via Deep Reinforcement Learning

Criticality-Guided Deep Reinforcement Learning for Motion Planning

Robotic Arm Motion Planning with Autonomous Obstacle Avoidance Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Trajectory Planning Under Uncertain Constraints

Receding-Horizon Reinforcement Learning Approach for Kinodynamic Motion Planning of Autonomous Vehicles

An Intelligent Robot Motion Planning Method and Application Via LPPO in Unknown Environment

Adaptive Dynamic Collision Checking Based Industrial Robot Motion Planning Algorithms

Efficient Robot Motion Planning Via Sampling and Optimization

Robotic Arm Motion Planning Based on Curriculum Reinforcement Learning

A Fast and Smooth Planning Framework for Autonomous Mobile Robot in Complex Environments

An Efficiently Convergent Deep Reinforcement Learning-Based Trajectory Planning Method for Manipulators in Dynamic Environments

Motion Planner with Fixed-Horizon Constrained Reinforcement Learning for Complex Autonomous Driving Scenarios

An Adaptive Multi-Rrt Approach for Robot Motion Planning