Abstract:Robotic motion planning in dense and dynamic indoor scenarios constantly challenges the researchers because of the motion unpredictability of obstacles. Recent progress in reinforcement learning enables robots to better cope with the dense and unpredictable obstacles by encoding complex features of the robot and obstacles into the encoders like the long-short term memory (LSTM). Then these features are learned by the robot using reinforcement learning algorithms, such as the deep Q network and asynchronous advantage actor critic algorithm. However, existing methods depend heavily on expert experiences to enhance the convergence speed of the networks by initializing them via imitation learning. Moreover, those approaches based on LSTM to encode the obstacle features are not always efficient and robust enough, therefore sometimes causing the network overfitting in training. This paper focuses on the advantage actor critic algorithm and introduces an attention-based actor critic algorithm with experience replay algorithm to improve the performance of existing algorithm from two perspectives. First, LSTM encoder is replaced by a robust encoder attention weight to better interpret the complex features of the robot and obstacles. Second, the robot learns from its past prioritized experiences to initialize the networks of the advantage actor-critic algorithm. This is achieved by applying the prioritized experience replay method, which makes the best of past useful experiences to improve the convergence speed. As results, the network based on our algorithm takes only around 15% and 30% experiences to get rid of the early-stage training without the expert experiences in cases with five and ten obstacles, respectively. Then it converges faster to a better reward with less experiences (near 45% and 65% of experiences in cases with ten and five obstacles respectively) when comparing with the baseline LSTM-based advantage actor critic algorithm. Our source code is freely available at the GitHub (https://github.com/CHUENGMINCHOU/AW-PER-A2C).

An Integrating Planning Actor-Adaptive Critic Reinforcement Learning Algorithm

Self-play Reinforcement Learning with Comprehensive Critic in Computer Games

Efficient Actor-Critic Algorithm with Hierarchical Model Learning and Planning

Autonomous underwater vehicle path planning based on actor-multi-critic reinforcement learning

Critic PI2: Master Continuous Planning via Policy Improvement with Path Integrals and Deep Actor-Critic Reinforcement Learning

Explorer-Actor-Critic: Better Actors for Deep Reinforcement Learning

An advantage actor-critic algorithm for robotic motion planning in dense and dynamic scenarios

An Actor-Critic Method for Simulation-Based Optimization

Selector-Actor-Critic and Tuner-Actor-Critic Algorithms for Reinforcement Learning

Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space

Model Reference Output Feedback Control Using Episodic Natural Actor-Critic

A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming

Actor-Director-Critic: A Novel Deep Reinforcement Learning Framework

A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots

Potential Field Guided Actor-Critic Reinforcement Learning

Attention-based advantage actor-critic algorithm with prioritized experience replay for complex 2-D robotic motion planning

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

An Advanced Actor-Critic Algorithm for Training Video Game AI

Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Framework

Adjust Planning Strategies to Accommodate Reinforcement Learning Agents