Abstract:Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.

From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration

A Unified BEV Model for Joint Learning of 3D Local Features and Overlap Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Multi-camera Calibration Based on Multi-view Registration Algorithm

Focus on BEV: Self-calibrated Cycle View Transformation for Monocular Birds-Eye-View Segmentation

Wide-Baseline Multi-Camera Calibration using Person Re-Identification

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

CalibRBEV: Multi-Camera Calibration Via Reversed Bird's-eye-view Representations for Autonomous Driving

High-precision Human Body Acquisition Via Multi-View Binocular Stereopsis

Multi-Camera Joint Self-Calibration from Observations of Pedestrians

Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization

OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection

BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation

BEV-SUSHI: Multi-Target Multi-Camera 3D Detection and Tracking in Bird's-Eye View

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks

BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignment

Progressive Temporal Transformer for Bird’s-Eye-View Camera Pose Estimation

Robust Plane-Based Calibration of Multiple Non-Overlapping Cameras

LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-frame 3D Human Pose Estimation

Multiple Human Association and Tracking from Egocentric and Complementary Top Views