Abstract:Extracting robust and discriminative local features from images plays a vital role for long term visual localization, whose challenges are mainly caused by the severe appearance differences between matching images due to the day-night illuminations, seasonal changes, and human activities. Existing solutions resort to jointly learning both keypoints and their descriptors in an end-to-end manner, leveraged on large number of annotations of point correspondence which are harvested from the structure from motion and depth estimation algorithms. While these methods show improved performance over non-deep methods or those two-stage deep methods, i.e., detection and then description, they are still struggled to conquer the problems encountered in long term visual localization. Since the intrinsic semantics are invariant to the local appearance changes, this paper proposes to learn semantic-aware local features in order to improve robustness of local feature matching for long term localization. Based on a state of the art CNN architecture for local feature learning, i.e., ASLFeat, this paper leverages on the semantic information from an off-the-shelf semantic segmentation network to learn semantic-aware feature maps. The learned correspondence-aware feature descriptors and semantic features are then merged to form the final feature descriptors, for which the improved feature matching ability has been observed in experiments. In addition, the learned semantics embedded in the features can be further used to filter out noisy keypoints, leading to additional accuracy improvement and faster matching speed. Experiments on two popular long term visual localization benchmarks (Aachen Day and Night v1.1, Robotcar Seasons) and one challenging indoor benchmark (InLoc) demonstrate encouraging improvements of the localization accuracy over its counterpart and other competitive methods.

Local Feature Descriptor Learning with Adaptive Siamese Network

3D LiDAR-Based Global Localization Using Siamese Neural Network

Discriminatively Learning for Representing Local Image Features with Quadruplet Model

Robust Angular Local Descriptor Learning

Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions

Learning a Local Feature Descriptor for 3D LiDAR Scans

Learning local descriptors with multi-level feature aggregation and spatial context pyramid

DHNet: Working Double Hard to Learn a Convolutional Neural Network-Based Local Descriptor

Adaptive Deconvolution-based stereo matching Net for Local Stereo Matching

Working hard to know your neighbor's margins: Local descriptor learning loss

Kernelized Subspace Pooling for Deep Local Descriptors.

MTLDesc: Looking Wider to Describe Better

Category-Aware Siamese Learning Network for Few-Shot Segmentation

Learning Semantic-Aware Local Features for Long Term Visual Localization

Siamese PointNet: 3D Head Pose Estimation with Local Feature Descriptor

A Simple Task-aware Contrastive Local Descriptor Selection Strategy for Few-shot Learning between inter class and intra class

Deep Unsupervised Binary Descriptor Learning Through Locality Consistency and Self Distinctiveness

Spatial-Adaptive and Feature-Enhanced Siamese Network for Change Detection.

Siamese Local and Global Networks for Robust Face Tracking

Embedded Spectral Descriptors: Learning the point-wise correspondence metric via Siamese neural networks

Deep Corner