Abstract:Recent feature descriptor research has witnessed tremendous progress with the development of the deep neural network. However, most existing descriptors solely focus on learning strong discriminativeness with deep-invariant features, neglecting their representation ability and rich hierarchical clues hidden in images, which could further establish high-quality matches via implicit hierarchical comparisons in rich representative descriptor space. In this article, we consider both the discriminative and representation ability of feature descriptors to enrich the descriptor space with a novel representative learning framework. On the one hand, we introduce histogram of oriented gradient (HOG) as a prior term to guide our descriptor to learn a powerful representation and robustness in a self-supervised manner. On the other hand, we present an adaptive triplet loss (ATL), which penalizes the triplet loss (TL) according to the descriptor matching distances in order to encourage our descriptor to learn strong discriminativeness. Moreover, to fully use the information encapsulated in images and boost the representation ability, we propose a novel HIerarchical Feature Transformer Network (HIFT), which derives dense descriptions from the semantic and cross-scale-enhanced hierarchical features in a local-to-global manner. Extensive experiments on popular feature matching and visual localization benchmarks show that the HIFT achieves highly competitive performance compared with the state-of-the-art methods. Applications on visual measurement tasks of visual 3-D reconstruction and ego-motion estimation also demonstrate the high generalization ability of our method. Our model is available at https://github.com/Ray2OUC/HIFT.

Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Investigating the Vision Transformer Model for Image Retrieval Tasks

Research on Image Classification Method of Features of Combinatorial Convolution

Learning Enriched Feature Descriptor for Image Matching and Visual Measurement

CNN Vs. SIFT for Image Retrieval: Alternative or Complementary?

CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

Affine-invariant SIFT Descriptor with Global Context

Computer Vision : History , the Rise of Deep Networks , and Future Vistas Panel on Perception and Cognition , MORS Meeting on Artificial Intelligence and Autonomy

Do Convnets Learn Correspondence?

Universal Correspondence Network

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

CDbin: Compact Discriminative Binary Descriptor Learned With Efficient Neural Network

Multi-scale Matching Networks for Semantic Correspondence

A New Framework for Feature Descriptor Based on SIFT

Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence

A Concurrent Multiscale Detector for End-to-End Image Matching

On the Large-Scale Transferability of Convolutional Neural Networks.

Learning General Descriptors for Image Matching with Regression Feedback

SIFT Matching by Context Exposed

An elliptical sampling based fast and robust feature descriptor for image matching

DHNet: Working Double Hard to Learn a Convolutional Neural Network-Based Local Descriptor