Abstract:Semantic navigation is necessary to deploy mobile robots in uncontrolled environments such as homes or hospitals. Many learning-based approaches have been proposed in response to the lack of semantic understanding of the classical pipeline for spatial navigation, which builds a geometric map using depth sensors and plans to reach point goals. Broadly, end-to-end learning approaches reactively map sensor inputs to actions with deep neural networks, whereas modular learning approaches enrich the classical pipeline with learning-based semantic sensing and exploration. However, learned visual navigation policies have predominantly been evaluated in sim, with little known about what works on a robot. We present a large-scale empirical study of semantic visual navigation methods comparing representative methods with classical, modular, and end-to-end learning approaches across six homes with no prior experience, maps, or instrumentation. We found that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% sim to a 23% real-world success rate because of a large image domain gap between sim and reality. For practitioners, we show that modular learning is a reliable approach to navigate to objects: Modularity and abstraction in policy design enable sim-to-real transfer. For researchers, we identify two key issues that prevent today's simulators from being reliable evaluation benchmarks—a large sim-to-real gap in images and a disconnect between sim and real-world error modes—and propose concrete steps forward.

Learning Navigation Costs from Demonstration with Semantic Observations

Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning

A LiDAR Based End to End Controller for Robot Navigation Using Deep Neural Network

Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

Predicting Dense and Context-aware Cost Maps for Semantic Robot Navigation

Navigation by Imitation in a Pedestrian-Rich Environment

Learning Social Navigation from Demonstrations with Conditional Neural Processes

Acquiring Robot Navigation Skill with Knowledge Learned from Demonstration

Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

Towards navigation without precise localization: Weakly supervised learning of goal-directed navigation cost map

Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics

Mapless Navigation With Safety-Enhanced Imitation Learning

No RL, No Simulation: Learning to Navigate without Navigating

Learning to Navigate in Indoor Environments: from Memorizing to Reasoning

Navigating to objects in the real world

Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning

Learning to Predict Navigational Patterns from Partial Observations

Improving Reliable Navigation under Uncertainty via Predictions Informed by Non-Local Information

QuasiNav: Asymmetric Cost-Aware Navigation Planning with Constrained Quasimetric Reinforcement Learning

Learning Social Navigation from Demonstrations with Deep Neural Networks