Abstract:Object pose estimation and camera localization are critical in various applications. However, achieving algorithm universality, which refers to category-level pose estimation and scene-independent camera localization, presents challenges for both techniques. Although the two tasks keep close relationships due to spatial geometry constraints, different tasks require distinct feature extractions. This paper pays attention to a unified RGB-D based framework that simultaneously performs category-level object pose estimation and scene-independent camera localization. The framework consists of a pose estimation branch called SLO-ObjNet, a localization branch called SLO-LocNet, a pose confidence calculation process and object-level optimization. At the start, we obtain the initial camera and object results from SLO-LocNet and SLO-ObjNet. In these two networks, we design there-level feature fusion modules as well as the loss function to achieve feature sharing between two tasks. Then the proposed approach involves a confidence calculation process to determine the accuracy of object poses obtained. Additionally, an object-level Bundle Adjustment (BA) optimization algorithm is further used to improve the precision of these techniques. The BA algorithm establishes relationships among feature points, objects, and cameras with the usage of camera-point, camera-object, and object-point metrics. To evaluate the performance of this approach, experiments are conducted on localization and pose estimation datasets including REAL275, CAMERA25, LineMOD, YCB-Video, 7 Scenes, ScanNet and TUM RGB-D. The results show that this approach outperforms existing methods in terms of both estimation and localization accuracy. Additionally, SLO-LocNet and SLO-ObjNet are trained on ScanNet data and tested on 7 Scenes and TUM RGB-D datasets to demonstrate its universality performance. Finally, we also highlight the positive effects of fusion modules, loss function, confidence process and BA for improving overall performance.

Discriminative Features Matter: Multi-layer Bilinear Pooling for Camera Localization

Fine-grained Visual Classification Via Multilayer Bilinear Pooling with Object Localization

3D LiDAR-Based Global Localization Using Siamese Neural Network

DeepCamera

Discriminatively Learning for Representing Local Image Features with Quadruplet Model

Grassmann Pooling As Compact Homogeneous Bilinear Pooling For Fine-Grained Visual Classification

Learning Bipartite Graph Matching for Robust Visual Localization.

Adaptive Bilinear Pooling for Fine-grained Representation Learning.

Combining Local and Global: Rich and Robust Feature Pooling for Visual Recognition.

DenserNet: Weakly Supervised Visual Localization Using Multi-Scale Feature Aggregation

Improved Bilinear CNN Model for Remote Sensing Scene Classification

Grouping Bilinear Pooling for Fine-Grained Image Classification

Kernelized Subspace Pooling for Deep Local Descriptors.

A Deep Feature Aggregation Network for Accurate Indoor Camera Localization

3DLG-Detector: 3D Object Detection via Simultaneous Local-Global Feature Learning

Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization

Rethinking Pooling for Multi-Granularity Features in Aerial-View Geo-Localization

Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization

Localizing Discriminative Visual Landmarks for Place Recognition

Locality-Constrained Bilinear Network for Land Cover Classification Using Heterogeneous Images

Cross-convolutional-layer Pooling for Generic Visual Recognition.