Abstract:The traditional CNN for 6D robot relocalization which outputs pose estimations does not interpret whether the model is making sensible predictions or just guessing at random. We found that convnet representations trained on classification problems generalize well to other tasks. Thus, we propose a multi-task CNN for robot relocalization, which can simultaneously perform pose regression and scene recognition. Scene recognition determines whether the input image belongs to the current scene in which the robot is located, not only reducing the error of relocalization but also making us understand with what confidence we can trust the prediction. Meanwhile, we found that when there is a large visual difference between testing images and training images, the pose precision becomes low. Based on this, we present the dual-level image-similarity strategy (DLISS), which consists of two levels: initial level and iteration-level. The initial level performs feature vector clustering in the training set and feature vector acquisition in testing images. The iteration level, namely, the PSO-based image-block selection algorithm, can select the testing images which are the most similar to training images based on the initial level, enabling us to gain higher pose accuracy in testing set. Our method considers both the accuracy and the robustness of relocalization, and it can operate indoors and outdoors in real time, taking at most 27 ms per frame to compute. Finally, we used the Microsoft 7Scenes dataset and the Cambridge Landmarks dataset to evaluate our method. It can obtain approximately 0.33 m and 7.51∘ accuracy on 7Scenes dataset, and get approximately 1.44 m and 4.83∘ accuracy on the Cambridge Landmarks dataset. Compared with PoseNet, our CNN reduced the average positional error by 25% and the average angular error by 27.79% on 7Scenes dataset, and reduced the average positional error by 40% and the average angular error by 28.55% on the Cambridge Landmarks dataset. We show that our multi-task CNN can localize from high-level features and is robust to images which are not in the current scene. Furthermore, we show that our multi-task CNN gets higher accuracy of relocalization by using testing images obtained by DLISS.

Night-time indoor relocalization using depth image with Convolutional Neural Networks

Indoor Relocalization in Challenging Environments with Dual-Stream Convolutional Neural Networks

Delving deeper into convolutional neural networks for camera relocalization.

A Cnn-Based Depth Estimation Approach With Multi-Scale Sub-Pixel Convolutions And A Smoothness Constraint

A new approach to train convolutional neural networks for real-time 6-DOF camera relocalization

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

VNLSTM-PoseNet: A Novel Deep ConvNet for Real-Time 6-DOF Camera Relocalization in Urban Streets.

Learn then Match: A Fast Coarse-to-fine Depth Image-Based Indoor Localization Framework for Dark Environments Via Deep Learning and Keypoint-Based Geometry Alignment

Indoor Visual Positioning Aided by CNN-Based Image Retrieval: Training-Free, 3D Modeling-Free

Visual Robot Relocalization Based on Multi-Task CNN and Image-Similarity Strategy

Scene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization

A 6-Dofs Event-Based Camera Relocalization System by CNN-LSTM and Image Denoising

Convolutional Neural Network-Based Coarse Initial Position Estimation Of A Monocular Camera In Large-Scale 3d Light Detection And Ranging Maps

Predicting World Coordinates of Pixels in RGB Images Using Convolutional Neural Network for Camera Relocalization

INDOOR LIDAR RELOCALIZATION BASED ON DEEP LEARNING USING A 3D MODEL

Monocular Indoor Depth Estimation Method Based on Neural Networks with Constraints on Two-Dimensional Images and Three-Dimensional Geometry

Image-similarity-based Convolutional Neural Network for Robot Visual Relocalization

Camera Re-localization by Training Multi-dataset Simultaneously Via Convolutional Neural Network

An improved lightweight deep neural network with knowledge distillation for local feature extraction and visual localization using images and LiDAR point clouds

Indoor Scene Recognition Mechanism Based on Direction-Driven Convolutional Neural Networks