Abstract:Reinforcement learning (RL) is a promising approach in unmanned ground vehicles (UGVs) applications, but limited computing resource makes it challenging to deploy a well-behaved RL strategy with sophisticated neural networks. Meanwhile, the training of RL on navigation tasks is difficult, which requires a carefully-designed reward function and a large number of interactions, yet RL navigation can still fail due to many corner cases. This shows the limited intelligence of current RL methods, thereby prompting us to rethink combining RL with human intelligence. In this paper, a human-guided RL framework is proposed to improve RL performance both during learning in the simulator and deployment in the real world. The framework allows humans to intervene in RL's control progress and provide demonstrations as needed, thereby improving RL's capabilities. An innovative human-guided RL algorithm is proposed that utilizes a series of mechanisms to improve the effectiveness of human guidance, including human-guided learning objective, prioritized human experience replay, and human intervention-based reward shaping. Our RL method is trained in simulation and then transferred to the real world, and we develop a denoised representation for domain adaptation to mitigate the simulation-to-real gap. Our method is validated through simulations and real-world experiments to navigate UGVs in diverse and dynamic environments based only on tiny neural networks and image inputs. Our method performs better in goal-reaching and safety than existing learning- and model-based navigation approaches and is robust to changes in input features and ego kinetics. Furthermore, our method allows small-scale human demonstrations to be used to improve the trained RL agent and learn expected behaviors online.

Reinforcement Learning-Based Visual Navigation With Information-Theoretic Regularization

Visual Navigation with Multiple Goals Based on Deep Reinforcement Learning

Target-driven Indoor Visual Navigation Using Inverse Reinforcement Learning

Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Skill-Based Hierarchical Reinforcement Learning for Target Visual Navigation

Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Learning Complicated Navigation Skills from Limited Experience Via Augmenting Offline Datasets

Autonomous Multi-View Navigation Via Deep Reinforcement Learning

Discovering Intrinsic Subgoals for Vision-and-Language Navigation via Hierarchical Reinforcement Learning

Target-driven Visual Navigation in Indoor Scenes Using Reinforcement Learning and Imitation Learning

Pose-guided End-to-end Visual Navigation

Learning Heterogeneous Relation Graph and Value Regularization Policy for Visual Navigation

A comparison of reinforcement learning models of human spatial navigation

Navigation for Autonomous Vehicles Via Fast-Stable and Smooth Reinforcement Learning

Learning Cross Dimension Scene Representation for Interactive Navigation Agents in Obstacle-Cluttered Environments

Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships

Auto-Navigator: Decoupled Neural Architecture Search for Visual Navigation

Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation

Active Perception for Visual-Language Navigation