Abstract:Camera relocalization is a challenging task to estimate camera pose within a known scene, with wide applications in the fields of Virtual Reality (VR), Augmented Reality (AR), robotics, and etc. Most existing learning-based methods invariably utilize all the information within an image for pose estimation. Although these methods have demonstrated leading pose accuracy in some cases, they are still far from being sufficient to handle the robustness under challenging viewpoints with less impacts on the localization accuracy for viewpoints that are easier to localize. In this paper, we propose a novel two-branch camera pose estimation framework: one branch utilizes keypoint-guided partial scene coordinate regression, while the other employs full scene coordinate regression to assess the credibility of image poses, thereby enabling more accurate camera localization. In particular, we devise a keypoint selection method predicated on matching rates which is designed to measure the matching quality between a 3D keypoint and 2D keypoints across views. With these selected 3D keypoints, we can generate 2D supervision mask with the ground-truth camera pose to supervise the keypoint prediction from the keypoint selection network. Meanwhile, we further refine the 2D supervision mask through the optimization with reprojection errors on the scene coordinate network, which estimates the scene coordinates for points within the scene that truly warrant attention, also enhances the localization performance. We also introduce a gated camera pose estimation strategy on the two-branch pose estimation framework, employing an updated keypoint selection network for images with higher credibility and a more robust network for difficult viewpoints. By adopting an effective curriculum learning scheme, we achieve higher accuracy within a training span of just 20 minutes. Our method's superior performance is validated through rigorous experimentation. The code is released at https://github.com/DUT-ICCD/KP-Guided-Reloc.

HR-APR: APR-agnostic Framework with Uncertainty Estimation and Hierarchical Refinement for Camera Relocalisation

KS-APR: Keyframe Selection for Robust Absolute Pose Regression

Absolute Camera Pose Regression Using an RGB-D Dual-Stream Network and Handcrafted Base Poses

Neural Refinement for Absolute Pose Regression with Feature Synthesis

Map-Relative Pose Regression for Visual Re-Localization

HRUNet: Assessing Uncertainty in Heart Rates Measured from Facial Videos

Camera Pose Auto-Encoders for Improving Pose Regression

DFNet: Enhance Absolute Pose Regression with Direct Feature Matching

Camera Pose Estimation with Uncertain Reference Point Based on Orthogonal Iterative

APR: Online Distant Point Cloud Registration Through Aggregated Point Cloud Reconstruction

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

Robust Keyframe-Based Monocular SLAM for Augmented Reality

Efficient Uncertainty Estimation for a Hybrid Wide-Area Human Action Analysis System

$α$-OCC: Uncertainty-Aware Camera-based 3D Semantic Occupancy Prediction

MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR

3D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose From Monocular Video

Unified Framework for the Joint Super-Resolution and Registration of Multiangle Multi/Hyperspectral Remote Sensing Images

Single Image based 3D Human Pose Estimation via Uncertainty Learning

Uni-SLAM: Uncertainty-Aware Neural Implicit SLAM for Real-Time Dense Indoor Scene Reconstruction

Exploring Matching Rates: from Keypoint Selection to Camera Relocalization

HAPOR: Hierarchical-Features Aligned Projection Optimization for Relocalization