Local Feature Matching Using Deep Learning: A Survey

Shibiao Xu,Shunpeng Chen,Rongtao Xu,Changwei Wang,Peng Lu,Li Guo

DOI: https://doi.org/10.1016/j.inffus.2024.102344

2024-03-11

Abstract:Local feature matching enjoys wide-ranging applications in the realm of computer vision, encompassing domains such as image retrieval, 3D reconstruction, and object recognition. However, challenges persist in improving the accuracy and robustness of matching due to factors like viewpoint and lighting variations. In recent years, the introduction of deep learning models has sparked widespread exploration into local feature matching techniques. The objective of this endeavor is to furnish a comprehensive overview of local feature matching methods. These methods are categorized into two key segments based on the presence of detectors. The Detector-based category encompasses models inclusive of Detect-then-Describe, Joint Detection and Description, Describe-then-Detect, as well as Graph Based techniques. In contrast, the Detector-free category comprises CNN Based, Transformer Based, and Patch Based methods. Our study extends beyond methodological analysis, incorporating evaluations of prevalent datasets and metrics to facilitate a quantitative comparison of state-of-the-art techniques. The paper also explores the practical application of local feature matching in diverse domains such as Structure from Motion, Remote Sensing Image Registration, and Medical Image Registration, underscoring its versatility and significance across various fields. Ultimately, we endeavor to outline the current challenges faced in this domain and furnish future research directions, thereby serving as a reference for researchers involved in local feature matching and its interconnected domains. A comprehensive list of studies in this survey is available at

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the problem of local feature matching in the field of computer vision, particularly in improving the accuracy and robustness of matching when facing challenges such as viewpoint changes and differences in lighting conditions. With the development of deep learning technology, research on local feature matching techniques has become increasingly widespread in recent years. The goal of the paper is to provide a comprehensive review, covering local feature matching methods based on detectors and those without detectors, and to analyze existing datasets and evaluation metrics for quantitative comparison of state-of-the-art techniques. Additionally, the paper explores the practical applications of local feature matching in various fields such as Structure from Motion (SfM), remote sensing image registration, and medical image registration, highlighting the challenges in current research and future research directions to provide reference for related researchers.

In short, the main issues addressed by the paper include:
1. **Improving the accuracy and robustness of local feature matching**: Especially under different viewpoints and lighting conditions.
2. **Reviewing existing methods**: Dividing local feature matching methods into two categories: detector-based and detector-free, and discussing the characteristics, advantages, and disadvantages of each method in detail.
3. **Evaluation and comparison**: Conducting quantitative comparisons of existing methods by analyzing commonly used datasets and evaluation metrics.
4. **Exploring practical applications**: Discussing the applications of local feature matching in various fields such as SfM, remote sensing image registration, and medical image registration.
5. **Proposing future research directions**: Identifying the challenges in current research and suggesting possible future research directions.