Abstract:Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.

Joint Multiview Segmentation And Localization Of Rgb-D Images Using Depth-Induced Silhouette Consistency

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Object Segmentation Ensuring Consistency Across Multi-Viewpoint Images

Robust 3D Reconstruction with an RGB-D Camera

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Simultaneous multi-body stereo and segmentation

Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization

An RGB-D Fusion Based Semantic Segmentation Algorithm Based on Neighborhood Metric Relations

Geometric Integration of Hybrid Correspondences for RGB-D Unidirectional Tracking

Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera

Accurate Rgb Camera Relocalization Using Regression Forest

Feature Regions Segmentation Based RGB-D Visual Odometry in Dynamic Environment

Joint Color-Spatial-Directional clustering and Region Merging (JCSD-RM) for unsupervised RGB-D image segmentation

Multiview Facial Landmark Localization in RGB-D Images Via Hierarchical Regression with Binary Patterns.

DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map

Towards Localizing Structural Elements: Merging Geometrical Detection with Semantic Verification in RGB-D Data

Local Supports Global: Deep Camera Relocalization With Sequence Enhancement

HGSLoc: 3DGS-based Heuristic Camera Pose Refinement

Robust Multiview Feature Learning for RGB-D Image Understanding

Geometry Depth Consistency in RGBD Relative Pose Estimation