Abstract:Semantic scene mapping is a challenge and significant task for robotic application, such as autonomous navigation and robot-environment interaction. In this paper, we propose a semantic pixel-wise mapping system for potential robotic applications. The system includes a novel spatio-temporal deep neural network for semantic segmentation and a Simultaneous Localisation and Mapping (SLAM) algorithm for 3D point cloud map. Their combination yields a 3D semantic pixel-wise map. The proposed network consists of Convolutional Neural Networks (CNNs) with two streams: spatial stream with images as the input and temporal stream with image differences as the input. Due to the use of both spatial and temporal information, it is called spatio-temporal deep neural network, which shows a better performance in both accuracy and robustness in semantic segmentation. Further, only keyframes are selected for semantic segmentation in order to reduce the computational burden for video streams and improve the real-time performance. Based on the result of semantic segmentation, a 3D semantic map is built up by using the 3D point cloud map from a SLAM algorithm. The proposed spatio-temporal neural network is evaluated on both Cityscapes benchmark (a public dataset) and Essex Indoor benchmark (a dataset we labelled ourselves manually). Compared with the state-of-the-art spatial only neural networks, the proposed network achieves better performances in both pixel-wise accuracy and Intersection over Union (IoU) for scene segmentation. The constructed 3D semantic map with our methods is accurate and meaningful for robotic applications.

Indoor Environment Semantic Topological Mapping Based on Deep Learning.

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

Real-time scene category of indoor robot based on semantic mapping

Visual Semantic Navigation Based on Deep Learning for Indoor Mobile Robots

Indoor Topological Localization Based on a Novel Deep Learning Technique

Semantic map for service robot navigation based on ROS

3d Semantic Mapping Based On Convolutional Neural Networks

Construction of Semantic Topological Map for Domestic Environment

Semantic Map Building Based on Object Detection for Indoor Navigation

An Approach for Construct Semantic Map with Scene Classification and Object Semantic Segmentation

Indoor Instance-Aware Semantic Mapping Using Instance Segmentation

Object-Oriented 3D Semantic Mapping Based on Instance Segmentation

Expression and Construction of a Multi-layer Map for Indoor Human-Robot Co-existing Environments

Semantic Scene Mapping with Spatio-temporal Deep Neural Network for Robotic Applications

Topological and Semantic Map Generation for Mobile Robot Indoor Navigation

Indoor 3D Semantic Robot VSLAM Based on Mask Regional Convolutional Neural Network

System of real time mobile robot semantic map building

Three-Dimensional Grid Semantic Map Building in Unstructured Indoor Environment

Object semantic map representation for indoor mobile robots

Map Building of Indoor Unknown Environment Based on Robot Service Mission Direction

What can i do around here? Deep functional scene understanding for cognitive robots