Abstract:Reinforcement learning (RL) is a promising approach in unmanned ground vehicles (UGVs) applications, but limited computing resource makes it challenging to deploy a well-behaved RL strategy with sophisticated neural networks. Meanwhile, the training of RL on navigation tasks is difficult, which requires a carefully-designed reward function and a large number of interactions, yet RL navigation can still fail due to many corner cases. This shows the limited intelligence of current RL methods, thereby prompting us to rethink combining RL with human intelligence. In this paper, a human-guided RL framework is proposed to improve RL performance both during learning in the simulator and deployment in the real world. The framework allows humans to intervene in RL's control progress and provide demonstrations as needed, thereby improving RL's capabilities. An innovative human-guided RL algorithm is proposed that utilizes a series of mechanisms to improve the effectiveness of human guidance, including human-guided learning objective, prioritized human experience replay, and human intervention-based reward shaping. Our RL method is trained in simulation and then transferred to the real world, and we develop a denoised representation for domain adaptation to mitigate the simulation-to-real gap. Our method is validated through simulations and real-world experiments to navigate UGVs in diverse and dynamic environments based only on tiny neural networks and image inputs. Our method performs better in goal-reaching and safety than existing learning- and model-based navigation approaches and is robust to changes in input features and ego kinetics. Furthermore, our method allows small-scale human demonstrations to be used to improve the trained RL agent and learn expected behaviors online.

Navigational Behavior of Humans and Deep Reinforcement Learning Agents

Predictive Hierarchical Reinforcement Learning for Path-Efficient Mapless Navigation with Moving Target.

A comparison of reinforcement learning models of human spatial navigation

Progress and Challenges in Applying Deep Reinforcement Learning to Intelligent Navigation

Sensor-Based Navigation Using Hierarchical Reinforcement Learning

Deep Reinforcement Learning for Navigation in AAA Video Games

A Role of Environmental Complexity on Representation Learning in Deep Reinforcement Learning Agents

Exploring the Potential of Deep Reinforcement Learning for Autonomous Navigation in Complex Environments

Learning to Navigate in Complex Environments

Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shaping

Deep reinforcement learning based mapless navigation for industrial AMRs: advancements in generalization via potential risk state augmentation

End-to-End Navigation Strategy with Deep Reinforcement Learning for Mobile Robots

An Efficient Deep Reinforcement Learning Algorithm for Mapless Navigation with Gap-Guided Switching Strategy

RL-DOVS: Reinforcement Learning for Autonomous Robot Navigation in Dynamic Environments

Deep reinforcement learning from human preferences

Mobile robot navigation based on Deep Reinforcement Learning: A brief review

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning

Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation

B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving

Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning