Abstract:Autonomously exploring and mapping unknown indoor scenes is a prerequisite for many robotic tasks. Deep reinforcement learning-based methods can teach a robot to leverage structural regularities of indoor scenes via interaction with the environment. Therefore, an efficient and robust exploration strategy is assured. The existing approaches either directly control the robot's motions, which leads to long decision sequences and necessitates an enormous number of training samples, or indirectly specify a long-term goal but do not guarantee the goal's reachability, which hinders the training as well. To overcome these problems, we propose a novel autonomous scene exploration method, using an experience enhancement algorithm to accelerate policy training and generate highly efficient exploratory targets. We first incorporate an off-policy reinforcement learning algorithm and the experience replay buffer mechanism. Then, a global exploration policy is used to specify a long-term goal. Next, we take advantage of an incremental heuristic pathfinding algorithm to plan a collision-avoidance path to the long-term goal. When the robot follows the path, we divide it into several segments and evaluate the subpath rewards, which help further improve existing experiences. Finally, the experiences are filtered based on the temporal difference error and stored in the experience replay buffer. The proposed method can extract correct behaviors from past failures and designate accessible long-term goals. Through various experiments, we validated that our approach can effectively improve the training efficiency of the global exploration policy and the final performance of the overall system.

A Novel Experience-Based Exploration Method for Q-Learning.

Gradient Q : A Unified Algorithm with Function Approximation for Reinforcement Learning

Learning biped locomotion based on Q-learning and neural networks

An Improved Q-learning Algorithm Based on Exploration Region Expansion Strategy

An Optimized Q-Learning Algorithm Based on the Thinking of Tabu Search

Topological Experience Replay

DQN with model-based exploration: efficient learning on environments with sparse rewards

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Mimicking Human Intuition: Cognitive Belief-Driven Q-Learning

Q_Learning Based on Active Backup and Memory Mechanism

Autonomous Exploration for Mobile Robot using Q-learning

A Novel Multi-Step Q-learning Method to Improve Data Efficiency for Deep Reinforcement Learning.

Iterated $Q$-Network: Beyond One-Step Bellman Updates in Deep Reinforcement Learning

A novel multi-agent Q-learning algorithm in cooperative multi-agent system

Learning to explore by reinforcement over high-level options

Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples

Mobile robot navigation method based on improved Q-learning algorithm

Dynamic Subgoal-based Exploration via Bayesian Optimization

Autonomous Scene Exploration Using Experience Enhancement

A Novel Update Mechanism for Q-Networks Based On Extreme Learning Machines

Multi-Agent Exploration Via Self-Learning and Social Learning