Abstract:Compared with such high-precision and dense representation maps as the point cloud or occupancy grid maps, the landmark map has the advantages of compactness and memory efficiency, where these advantages are particularly prominent in a large-scale environment for robot localization, navigation, or environment measurement. However, training a robot to identify and select useful landmarks for localization is challenging. Due to this limitation, most landmarks are identified using handcrafted features. In this article, we propose a multitask neural network called LandmarkNet to build a flexible, optimal, and interpretable landmark map. The multitask neural network with an attention mechanism is trained for the robot's pose regression and the semantic segmentation of images from a set of sequences of monocular images. The image segmentation yields auxiliary semantic information for landmark selection based on some common-sense notions such as that the sky and clouds cannot be used as landmarks for robot localization. The attention feature map thus learned is used to select landmarks for robot localization based on the optimal projection between the image sequence with the corresponding pose of the robot and the semantic segmentation of the images. The learned landmark maps are also applied to mobile robot localization using traditional Monte Carlo localization (MCL). To verify the validity of our methods, we performed experiments on simulation, an open dataset (KITTI monoVO), and a dataset that we had created on the campus of Tongji University. The results show that the learned landmark map, which was only 6% of the size of the original map, can be used to accurately perform visual localization tasks. We open the source code on website https://github.com/dotstudio01/landmark_learning.git.

Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization

Long-Term Map-Based Visual Localization: Analysis of Individual Components of a Hierarchical Pipeline

LODM: Large-scale Online Dense Mapping for UAV

Representing 3D sparse map points and lines for camera relocalization

Visual Landmark Learning Via Attention-Based Deep Neural Networks.

Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation

A Deep Learning-Based Visual Map Generation for Mobile Robot Navigation

Long-term Visual Map Sparsification with Heterogeneous GNN

Learning to Navigate using Visual Sensor Networks

Simultaneous Localization and Mapping in A Camera Network

Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching

Active Neural Localization

Deep Learning for Visual Localization and Mapping: A Survey

Sparse Image based Navigation Architecture to Mitigate the need of precise Localization in Mobile Robots

ERF-IMCS: an Efficient and Robust Framework with Image-Based Monte Carlo Scheme for Indoor Topological Navigation

VectorMapNet: End-to-end Vectorized HD Map Learning

DMRL-Net: Differentiable Multi-view Representation Learning Network.

D2S: Representing sparse descriptors and 3D coordinates for camera relocalization

VRS-NeRF: Visual Relocalization with Sparse Neural Radiance Field