Abstract:Autonomously exploring and mapping unknown indoor scenes is a prerequisite for many robotic tasks. Deep reinforcement learning-based methods can teach a robot to leverage structural regularities of indoor scenes via interaction with the environment. Therefore, an efficient and robust exploration strategy is assured. The existing approaches either directly control the robot's motions, which leads to long decision sequences and necessitates an enormous number of training samples, or indirectly specify a long-term goal but do not guarantee the goal's reachability, which hinders the training as well. To overcome these problems, we propose a novel autonomous scene exploration method, using an experience enhancement algorithm to accelerate policy training and generate highly efficient exploratory targets. We first incorporate an off-policy reinforcement learning algorithm and the experience replay buffer mechanism. Then, a global exploration policy is used to specify a long-term goal. Next, we take advantage of an incremental heuristic pathfinding algorithm to plan a collision-avoidance path to the long-term goal. When the robot follows the path, we divide it into several segments and evaluate the subpath rewards, which help further improve existing experiences. Finally, the experiences are filtered based on the temporal difference error and stored in the experience replay buffer. The proposed method can extract correct behaviors from past failures and designate accessible long-term goals. Through various experiments, we validated that our approach can effectively improve the training efficiency of the global exploration policy and the final performance of the overall system.

ACDER: Augmented Curiosity-Driven Experience Replay

Efficient Diversity-based Experience Replay for Deep Reinforcement Learning

AHEGC: Adaptive Hindsight Experience Replay with Goal-Amended Curiosity Module for Robot Control.

Soft Hindsight Experience Replay

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

Relay Hindsight Experience Replay: Self-guided continual reinforcement learning for sequential object manipulation tasks with sparse rewards

Enhancing Curiosity-driven Reinforcement Learning Through Historical State Information for Long-term Exploration

Advances in Experience Replay

Exploration via Hindsight Goal Generation

Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency

Invariant Transform Experience Replay: Data Augmentation for Deep Reinforcement Learning.

Random curiosity-driven exploration in deep reinforcement learning

CIER: A Novel Experience Replay Approach with Causal Inference in Deep Reinforcement Learning

Consistent Experience Replay in High-Dimensional Continuous Control with Decayed Hindsights

Curiosity-tuned experience replay for wargaming decision modeling without reward-engineering

Replay across Experiments: A Natural Extension of Off-Policy RL

PIEDQN: an Active-exploration Framework Capable of Prioritizing Experience Replay in Autonomous Driving

Overcoming Exploration in Reinforcement Learning with Demonstrations

Invariant Transform Experience Replay.

Adaptive trajectory-constrained exploration strategy for deep reinforcement learning

Autonomous Scene Exploration Using Experience Enhancement