Abstract:Recently, direct visual localization with convolutional neural networks has attracted researchers' attention with achieving an end-to-end process. However, on the one side, the lack of using 3D information leads to imprecise accuracy. Meanwhile, the single input image confuses the relocalization in the scenes that keep similar views at different positions. On the other side, the relocalization problem in variable or dynamic scenes is still challenging. Concentrating on these concerns, we propose two multitask relocalization networks called MMLNet and MMLNet+ for obtaining the 6-DoF camera pose in static, variable and dynamic scenes. Firstly, addressing the dataset lack of variable scenes, we construct a variable scene dataset with a semiautomatic process combining SFM and MVS algorithms with a few manual labels. Based on the process, three scenes covering an office, a bedroom and a sitting room are gathered and generated. Secondly, to enhance the perception between 2D images and 3D poses, we design a multitask network called MMLNet that regresses both camera pose and scene point cloud. Meanwhile, the Chamfer distance is joined into the original pose loss to optimize MMLNet. Moreover, MMLNet learns the pose trajectory feature by using LSTM layers to the additional pose array input, which meanwhile breaks through the limitation of single image input. Based on the MMLNet, aiming at dynamic and variable scenes, MMLNet+ outputs the auxiliary segmentation branch that distinguishes fixed, changeable or dynamic parts of the input image. Furthermore, we define the feature fusion block to implement the feature sharing among three tasks, further promoting the performance in dynamic and variable environments. Finally, experiments on static, dynamic and our constructed variable datasets demonstrate state-of-the-art relocalization performances of MMLNet and MMLNet+. Simultaneously, the positive effects of the pose learning part, reconstruction branch and segmentation task are also illustrated.

Deep Convolutional Neural Network for 6-DOF Image Localization

Normalized Localization for 6-DOF Camera Pose Regression

6-DOF Image Localization From Massive Geo-Tagged Reference Images.

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

Fast Estimation of Relative Poses for 6-DOF Image Localization

6D Object Pose Regression Via Supervised Learning on Point Clouds.

Deep Learning-Based 6-DoF Object Pose Estimation Considering Synthetic Dataset

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

Accurate Rgb Camera Relocalization Using Regression Forest

Semantic Segmentation and 6DoF Pose Estimation using RGB-D Images and Deep Neural Networks

A 6-Dofs Event-Based Camera Relocalization System by CNN-LSTM and Image Denoising

Deep Camera Pose Regression Using Pseudo-LiDAR

Relative geometry-aware siamese neural network for 6DOF camera relocalization

RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Images

Real-time Image-based 6-DOF Localization in Large-Scale Environments

InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

DeepIM: Deep Iterative Matching for 6D Pose Estimation

Deep 6-DoF camera relocalization in variable and dynamic scenes by multitask learning

Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image

6D pose estimation of 3D objects in scenes with mutual similarities and occlusions

A Novel 6D Pose Estimation Method for Indoor Objects Based on Monocular Regression Depth