Abstract:Robotic motion planning in dense and dynamic indoor scenarios constantly challenges the researchers because of the motion unpredictability of obstacles. Recent progress in reinforcement learning enables robots to better cope with the dense and unpredictable obstacles by encoding complex features of the robot and obstacles into the encoders like the long-short term memory (LSTM). Then these features are learned by the robot using reinforcement learning algorithms, such as the deep Q network and asynchronous advantage actor critic algorithm. However, existing methods depend heavily on expert experiences to enhance the convergence speed of the networks by initializing them via imitation learning. Moreover, those approaches based on LSTM to encode the obstacle features are not always efficient and robust enough, therefore sometimes causing the network overfitting in training. This paper focuses on the advantage actor critic algorithm and introduces an attention-based actor critic algorithm with experience replay algorithm to improve the performance of existing algorithm from two perspectives. First, LSTM encoder is replaced by a robust encoder attention weight to better interpret the complex features of the robot and obstacles. Second, the robot learns from its past prioritized experiences to initialize the networks of the advantage actor-critic algorithm. This is achieved by applying the prioritized experience replay method, which makes the best of past useful experiences to improve the convergence speed. As results, the network based on our algorithm takes only around 15% and 30% experiences to get rid of the early-stage training without the expert experiences in cases with five and ten obstacles, respectively. Then it converges faster to a better reward with less experiences (near 45% and 65% of experiences in cases with ten and five obstacles respectively) when comparing with the baseline LSTM-based advantage actor critic algorithm. Our source code is freely available at the GitHub (https://github.com/CHUENGMINCHOU/AW-PER-A2C).

Path-Analysis-Based Reinforcement Learning Algorithm for Imitation Filming

Imitation Learning-Based Algorithm for Drone Cinematography System

One-Shot Imitation Drone Filming of Human Motion Videos

Learning to Capture a Film-Look Video with a Camera Drone

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments

Attention-Based Deep Reinforcement Learning for Virtual Cinematography of 360 Videos

Learning to Drive Like Human Beings: A Method Based on Deep Reinforcement Learning

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Keyframe-Focused Visual Imitation Learning

Learning to Film from Professional Human Motion Videos

Attention-based advantage actor-critic algorithm with prioritized experience replay for complex 2-D robotic motion planning

Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving

Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator.

Imitation Learning of Robotic Arm with Hierarchical Training Based on Human Videos

A structured prediction approach for robot imitation learning

Accelerating Human Motion Imitation with Interactive Reinforcement Learning

Example-driven Virtual Cinematography by Learning Camera Behaviors

Act: An Autonomous Drone Cinematography System For Action Scenes

Imitation Learning-Based Drone Motion Planning in Dense Obstacle Scenarios

Automated Action Evaluation for Robotic Imitation Learning via Siamese Neural Networks