Abstract:Camera relocalization is a challenging task to estimate camera pose within a known scene, with wide applications in the fields of Virtual Reality (VR), Augmented Reality (AR), robotics, and etc. Most existing learning-based methods invariably utilize all the information within an image for pose estimation. Although these methods have demonstrated leading pose accuracy in some cases, they are still far from being sufficient to handle the robustness under challenging viewpoints with less impacts on the localization accuracy for viewpoints that are easier to localize. In this paper, we propose a novel two-branch camera pose estimation framework: one branch utilizes keypoint-guided partial scene coordinate regression, while the other employs full scene coordinate regression to assess the credibility of image poses, thereby enabling more accurate camera localization. In particular, we devise a keypoint selection method predicated on matching rates which is designed to measure the matching quality between a 3D keypoint and 2D keypoints across views. With these selected 3D keypoints, we can generate 2D supervision mask with the ground-truth camera pose to supervise the keypoint prediction from the keypoint selection network. Meanwhile, we further refine the 2D supervision mask through the optimization with reprojection errors on the scene coordinate network, which estimates the scene coordinates for points within the scene that truly warrant attention, also enhances the localization performance. We also introduce a gated camera pose estimation strategy on the two-branch pose estimation framework, employing an updated keypoint selection network for images with higher credibility and a more robust network for difficult viewpoints. By adopting an effective curriculum learning scheme, we achieve higher accuracy within a training span of just 20 minutes. Our method's superior performance is validated through rigorous experimentation. The code is released at https://github.com/DUT-ICCD/KP-Guided-Reloc.

Building Reliable Keypoint Matches by a Cascade of Classifiers with Resurrection Mechanism

An Accurate and Robust Registration Framework Based on Outlier Removal and Feature Point Adjustment for Remote Sensing Images

Region-restricted Rapid Keypoint Registration.

A Robust Background Registration Method Based on Multi Match Points

An Automatic Registration Approach to Laser Point Sets Based on Multidiscriminant Parameter Extraction

A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints

Unsupervised Non-rigid Histological Image Registration Guided by Keypoint Correspondences Based on Learnable Deep Features with Iterative Training

Robust Point Set Registration Method Based on Global Structure and Local Constraints

Bones Can't Be Triangles: Accurate and Efficient Vertebrae Keypoint Estimation through Collaborative Error Revision

Exploring Matching Rates: from Keypoint Selection to Camera Relocalization

Point-Based Registration for Multi-stained Histology Images

MSSF: A Novel Mutual Structure Shift Feature for Removing Incorrect Keypoint Correspondences between Images

Registration-Is-Evaluation: Robust Point Set Matching With Multigranular Prior Assessment

A new method for point cloud registration: Adaptive relation-oriented convolution and recurrent correspondence-walk

Robust Semantic Map Matching Algorithm Based on Probabilistic Registration Model

Rethinking of learning-based 3D keypoints detection for large-scale point clouds registration

A Keypoint Matching Method Based on Hierarchical Learning

Registration-reliability Based Strategy to Enhance Multi-Frame Super-Resolution Algorithms

Multilevel Attention Siamese Network for Keypoint Detection in Optical and SAR Images

Reliable and stable fundus image registration based on brain-inspired spatially-varying adaptive pyramid context aggregation network

Keypoint Matching for Point Cloud Registration Using Multiplex Dynamic Graph Attention Networks