Abstract:Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km2 area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the issue of feature point matching in large-scale 3D scene reconstruction using Structure-from-Motion (SfM) technology. Specifically, the paper proposes an improved method to select stable feature points to enhance matching speed and quality. #### Main Issues 1. **Large Number of Feature Points**: Existing algorithms (such as Scale-Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF)) extract a very large number of feature points, with a single image potentially containing hundreds of thousands of feature points. This significantly reduces the efficiency of image matching and may lead to failures in aerial triangulation. 2. **Uneven Quality of Feature Points**: The quality of feature points extracted by existing methods is inconsistent. Feature points with poor stability can drag down the matching rate and the accuracy of aerial triangulation. #### Solution The paper proposes a new method that selects more stable and reliable feature points by considering the semantic and structural characteristics of the image. The specific steps are as follows: 1. **Vegetation Area Recognition and Line Feature Extraction**: Use the Visible Band Difference Vegetation Index (VDVI) to identify vegetation areas and optimize the Line Segment Detector (LSD) algorithm to extract line features. 2. **Constructing a Binary Classification Model**: Use the results of vegetation area recognition as semantic constraints and the results of line feature extraction as structural constraints to establish a feature point classification model, categorizing feature points into three types. 3. **Progressive Feature Point Selection Algorithm**: Select feature points in the DoG pyramid by category and level until the feature point quantity threshold is met. ### Experimental Validation Experimental results show that compared to existing methods, the proposed method not only effectively reduces the number of feature points but also better reflects the scene structure. Additionally, the average reprojection error of aerial triangulation is reduced by 20%, the feature point matching rate is increased by 3%, and the selected feature points are more stable and reasonable. ### Conclusion The method proposed in the paper significantly improves the quality of feature matching while ensuring matching efficiency, addressing the shortcomings of existing methods in large-scale oblique photogrammetry 3D reconstruction.

An Improved Method for Stable Feature Points Selection in Structure-from-Motion Considering Image Semantic and Structural Characteristics

An optimization method for registration and mosaicking of remote sensing images

Real-time and High Precision Feature Matching Between Blur Aerial Images

Improve Matching Results for Structure from Motion Problems.

Efficient Non-Consecutive Feature Tracking For Structure-From-Motion

TC-SfM: Robust Track-Community-Based Structure-from-Motion

An Accurate and Robust Registration Framework Based on Outlier Removal and Feature Point Adjustment for Remote Sensing Images

Efficient Non-Consecutive Feature Tracking for Robust Structure-From-Motion

Leveraging photogrammetric mesh models for aerial-ground feature point matching toward integrated 3D reconstruction

A New Method to Extract Stable Feature Points Based on Self-Generated Simulation Images

Evaluation of Local Features for Structure from Motion.

Feature-Point Matching for Aerial and Ground Images by Exploiting Line Segment-Based Local-Region Constraints

IMPROVING PERFORMANCE OF FEATURE EXTRACTION IN SFM ALGORITHMS FOR 3D SPARSE POINT CLOUD

Efficient and Robust Large-Scale Structure-from-motion Via Track Selection and Camera Prioritization

A Discriminative Point Matching Algorithm Based on Local Structure Consensus Constraint

An Efficient Visual SfM Framework Using Planar Markers.

Large-Scale Structure from Motion with Semantic Constraints of Aerial Images

Exploiting Local Linear Geometric Structure for Identifying Correct Matches

A Fast and Accurate Purification Method of Image Feature Point Pairs Using Structural Consistency Constraints

Robust Incremental Structure-from-Motion with Hybrid Features

Efficient SfM for Large-Scale UAV Images Based on Graph-Indexed BoW and Parallel-Constructed BA Optimization