Abstract:Local feature extraction plays a crucial role in numerous critical visual tasks. However, there remains room for improvement in both descriptors and keypoints, particularly regarding the discriminative power of descriptors and the localization precision of keypoints. To address these challenges, this study introduces a novel local feature extraction pipeline named OSDFeat (Object and Spatial Discrimination Feature). OSDFeat employs a decoupling strategy, training descriptor and detection networks independently. Inspired by semantic correspondence, we propose an Object and Spatial Discrimination ResUNet (OSD-ResUNet). OSD-ResUNet captures features from the feature map that differentiate object appearance and spatial context, thus enhancing descriptor performance. To further improve the discriminative capability of descriptors, we propose a Discrimination Information Retained Normalization module (DIRN). DIRN complementarily integrates spatial-wise normalization and channel-wise normalization, yielding descriptors that are more distinguishable and informative. In the detection network, we propose a Cross Saliency Pooling module (CSP). CSP employs a cross-shaped kernel to aggregate long-range context in both vertical and horizontal dimensions. By enhancing the saliency of keypoints, CSP enables the detection network to effectively utilize descriptor information and achieve more precise localization of keypoints. Compared to the previous best local feature extraction methods, OSDFeat achieves Mean Matching Accuracy of 79.4% in local feature matching task, improving by 1.9% and achieving state-of-the-art results. Additionally, OSDFeat achieves competitive results in Visual Localization and 3D Reconstruction. The results of this study indicate that object and spatial discrimination can improve the accuracy and robustness of local feature, even in challenging environments. The code is available at https://github.com/pandaandyy/OSDFeat.

Local feature matching from detector-based to detector-free: a survey

Local Feature Matching Using Deep Learning: A Survey

Local Feature Descriptor for Image Matching: A Survey

Image Matching from Handcrafted to Deep Features: A Survey

A survey of feature matching methods

Feature detection and description for image matching: from hand-crafted design to deep learning

Learning Local Event-based Descriptor for Patch-based Stereo Matching

Local Feature Detectors, Descriptors, and Image Representations: A Survey

Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work

Image Matching and Localization Based on Fusion of Handcrafted and Deep Features

Object and spatial discrimination makes weakly supervised local feature better

UAV image matching from handcrafted to deep local features

A Survey on Facial Features Localization

LoFTR: Detector-Free Local Feature Matching with Transformers

A survey and performance evaluation of deep learning methods for small object detection

Feature matching based on local windows aggregation

Detector-Free Feature Matching for Optical and SAR Images Based on a Two-Step Strategy

Learning Semantic-Aware Local Features for Long Term Visual Localization

From handcrafted to deep local features