Abstract:Robotic motion planning in dense and dynamic indoor scenarios constantly challenges the researchers because of the motion unpredictability of obstacles. Recent progress in reinforcement learning enables robots to better cope with the dense and unpredictable obstacles by encoding complex features of the robot and obstacles into the encoders like the long-short term memory (LSTM). Then these features are learned by the robot using reinforcement learning algorithms, such as the deep Q network and asynchronous advantage actor critic algorithm. However, existing methods depend heavily on expert experiences to enhance the convergence speed of the networks by initializing them via imitation learning. Moreover, those approaches based on LSTM to encode the obstacle features are not always efficient and robust enough, therefore sometimes causing the network overfitting in training. This paper focuses on the advantage actor critic algorithm and introduces an attention-based actor critic algorithm with experience replay algorithm to improve the performance of existing algorithm from two perspectives. First, LSTM encoder is replaced by a robust encoder attention weight to better interpret the complex features of the robot and obstacles. Second, the robot learns from its past prioritized experiences to initialize the networks of the advantage actor-critic algorithm. This is achieved by applying the prioritized experience replay method, which makes the best of past useful experiences to improve the convergence speed. As results, the network based on our algorithm takes only around 15% and 30% experiences to get rid of the early-stage training without the expert experiences in cases with five and ten obstacles, respectively. Then it converges faster to a better reward with less experiences (near 45% and 65% of experiences in cases with ten and five obstacles respectively) when comparing with the baseline LSTM-based advantage actor critic algorithm. Our source code is freely available at the GitHub (https://github.com/CHUENGMINCHOU/AW-PER-A2C).

Path planning of robotic arm based on deep reinforcement learning algorithm

Robotic Arm Motion Planning Based on Curriculum Reinforcement Learning

Robotic Arm Motion Planning with Autonomous Obstacle Avoidance Based on Deep Reinforcement Learning

A Path Planning Method Based on Deep Reinforcement Learning with Improved Prioritized Experience Replay for Human-Robot Collaboration

Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor–Critic with Hindsight Experience Replay

Deep Reinforcement Learning for an Anthropomorphic Robotic Arm under Sparse Reward Tasks

Humanoid motion planning of robotic arm based on human arm action feature and reinforcement learning

Robotic arm reinforcement learning control method based on autonomous visual perception

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Energy-Efficient Motion Planning and Control for Robotic Arms Via Deep Reinforcement Learning

Multi-Objective Optimal Trajectory Planning for Robotic Arms Using Deep Reinforcement Learning

The Robotic Arm Velocity Planning Based on Reinforcement Learning

Attention-based advantage actor-critic algorithm with prioritized experience replay for complex 2-D robotic motion planning

Reinforcement learning path planning method incorporating multi-step Hindsight Experience Replay for lightweight robots

Mobile Robot Path Planning Method Based on Deep Reinforcement Learning Algorithm

Deep-Reinforcement-Learning-based Path Planning for Industrial Robots using Distance Sensors as Observation

Robot Arm Simulation Based on Model-Free Reinforcement Learning

Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Objects Grasping

Real-time Motion Planning for Robotic Teleoperation Using Dynamic-goal Deep Reinforcement Learning

Robot Arm Reaching Based on Inner Rehearsal

Research on Complex Robot Manipulation Tasks Based on Hindsight Trust Region Policy Optimization