Abstract:Reinforcement learning (RL) is a promising approach in unmanned ground vehicles (UGVs) applications, but limited computing resource makes it challenging to deploy a well-behaved RL strategy with sophisticated neural networks. Meanwhile, the training of RL on navigation tasks is difficult, which requires a carefully-designed reward function and a large number of interactions, yet RL navigation can still fail due to many corner cases. This shows the limited intelligence of current RL methods, thereby prompting us to rethink combining RL with human intelligence. In this paper, a human-guided RL framework is proposed to improve RL performance both during learning in the simulator and deployment in the real world. The framework allows humans to intervene in RL's control progress and provide demonstrations as needed, thereby improving RL's capabilities. An innovative human-guided RL algorithm is proposed that utilizes a series of mechanisms to improve the effectiveness of human guidance, including human-guided learning objective, prioritized human experience replay, and human intervention-based reward shaping. Our RL method is trained in simulation and then transferred to the real world, and we develop a denoised representation for domain adaptation to mitigate the simulation-to-real gap. Our method is validated through simulations and real-world experiments to navigate UGVs in diverse and dynamic environments based only on tiny neural networks and image inputs. Our method performs better in goal-reaching and safety than existing learning- and model-based navigation approaches and is robust to changes in input features and ego kinetics. Furthermore, our method allows small-scale human demonstrations to be used to improve the trained RL agent and learn expected behaviors online.

How To Train Your HERON

A Simulator and First Reinforcement Learning Results for Underwater Mapping

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

HER-PDQN: A Reinforcement Learning Approach for UAV Navigation with Hybrid Action Spaces and Sparse Rewards

Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation

Sensor-Based Navigation Using Hierarchical Reinforcement Learning

Reinforcement Learning-Based Cooperative Hunting for an Unmanned Surface Vehicle Swarm

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

Parallel Distributional Deep Reinforcement Learning for Mapless Navigation of Terrestrial Mobile Robots

Environment as Policy: Learning to Race in Unseen Tracks

Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots

Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality

How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned

NavREn-Rl: Learning to fly in real environment via end-to-end deep reinforcement learning using monocular images

Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Active Perception Applied To Unmanned Aerial Vehicles Through Deep Reinforcement Learning