Abstract:Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.

Multi-Scale Tetrahedral Fusion Of A Similarity Reconstruction And Noisy Positional Measurements

Two-Stage Multi-Camera Constrain Mapping Pipeline for Large-Scale 3D Reconstruction

MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

A Multiscale-Contour-based Interpolation Framework for Generating a Time-Varying Quasi-Dense Point Cloud Sequence.

APPTracker Plus : Displacement Uncertainty for Occlusion Handling in Low-Frame-Rate Multiple Object Tracking

Robust Metric Reconstruction from Challenging Video Sequences

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

Efficient Map Fusion for Multiple Implicit SLAM Agents

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

Intensity Video Guided 4D Fusion for Improved Highly Dynamic 3D Reconstruction

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras.

Hybrid Iteration and Optimization-based Three-dimensional Reconstruction for Space Non-Cooperative Targets with Monocular Vision and Sparse Lidar Fusion

HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors.

FROMFusion: Fast and Robust On-Manifold Dense Reconstruction for Low-Cost Wheeled Robots

MCSfM: Multi-Camera-Based Incremental Structure-From-Motion

Noisy Depth Maps Fusion for Multiview Stereo Via Matrix Completion

Parallel Structure from Motion from Local Increment to Global Averaging

Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization