Combining Absolute and Semi-Generalized Relative Poses for Visual Localization

Vojtech Panek,Torsten Sattler,Zuzana Kukelova

2024-09-22

Abstract:Visual localization is the problem of estimating the camera pose of a given query image within a known scene. Most state-of-the-art localization approaches follow the structure-based paradigm and use 2D-3D matches between pixels in a query image and 3D points in the scene for pose estimation. These approaches assume an accurate 3D model of the scene, which might not always be available, especially if only a few images are available to compute the scene representation. In contrast, structure-less methods rely on 2D-2D matches and do not require any 3D scene model. However, they are also less accurate than structure-based methods. Although one prior work proposed to combine structure-based and structure-less pose estimation strategies, its practical relevance has not been shown. We analyze combining structure-based and structure-less strategies while exploring how to select between poses obtained from 2D-2D and 2D-3D matches, respectively. We show that combining both strategies improves localization performance in multiple practically relevant scenarios.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the problem of how to improve localization accuracy in visual localization tasks by combining structure-based methods and structure-less methods. Specifically:

1. **Comparison of Existing Methods**: Traditional structure-based methods rely on accurate 3D models, while structure-less methods use 2D-2D matching, which do not depend on 3D models but usually have lower accuracy.
2. **Effectiveness of Adaptation Strategies**: The researchers analyzed how to choose pose estimation results obtained from 2D-2D matching and 2D-3D matching in different scenarios, and demonstrated the effectiveness of this combined strategy in practical applications.
3. **Importance of Scoring Functions**: The paper emphasizes the importance of selecting an appropriate scoring function to choose the best pose from the two different types of matching, and explores several simple scoring strategies.

Through experimental validation, it is shown that combining these two methods can significantly improve performance in cases where the database images are sparse or the 3D geometric structure is inaccurate.