Abstract:Semantic 3D mapping is one of the most important fields in robotics, and has been used in many applications, such as robot navigation, surveillance, and virtual reality. In general, semantic 3D mapping is mainly composed of 3D reconstruction and semantic segmentation. As these technologies evolve, there has been great progress in semantic 3D mapping in recent years. Furthermore, the number of robotic applications requiring semantic information in 3D mapping to perform high-level tasks has increased, and many studies on semantic 3D mapping have been published. Existing methods use a camera for both 3D reconstruction and semantic segmentation. However, this is not suitable for large-scale environments and has the disadvantage of high computational complexity. To address this problem, we propose a multimodal sensor-based semantic 3D mapping system using a 3D Lidar combined with a camera. In this study, the odometry is obtained by high-precision global positioning system (GPS) and inertial measurement unit (IMU), and it is estimated by iterative closest point (ICP) when a GPS signal is weak. Then, we use the latest 2D convolutional neural network (CNN) for semantic segmentation. To build a semantic 3D map, we integrate the 3D map with semantic information by using coordinate transformation and Bayes’ update scheme. In order to improve the semantic 3D map, we propose a 3D refinement process to correct wrongly segmented voxels and remove traces of moving vehicles in the 3D map. Through experiments on challenging sequences, we demonstrate that our method outperforms state-of-the-art methods in terms of accuracy and intersection over union (IoU). Thus, our method can be used for various applications that require semantic information in 3D map.

Autonomous mapping for robot based on monocular vision and DGSOM algorithm

3D Map Building for Mobile Robot under Unknown Outdoor Environments

Robust Indoor Localization and Map Matching Algorithm with Visual SLAM

A Computer Vision for Navigation of Mobile Robots

Environment Map Building and Localization for Robot Navigation Based on Image Sequences

Monocular Vision Based Obstacle Detection for Robot Navigation in Unstructured Environment

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

Learning Autonomous Exploration and Mapping with Semantic Vision

Large-Scale 3D Semantic Mapping Using Monocular Vision

DSOMF: A Dynamic Environment Simultaneous Localization and Mapping Technique Based on Machine Learning

Monocular Vision SLAM for Large Scale Outdoor Environment

Monocular Vision Navigation and Control of Mobile Robot

A survey of image semantics-based visual simultaneous localization and mapping: Application-oriented solutions to autonomous navigation of mobile robots

Dynamic Map Building for an Autonomous Mobile Robot

Efficient Exploration for Real-Time Robot Indoor 3d Mapping

Development of an indoor localization and navigation system based on monocular SLAM for mobile robots

Hybrid Semi-Dense 3D Semantic-Topological Mapping From Stereo Visual-Inertial Odometry SLAM With Loop Closure Detection

Vision-based autonomous mapping and exploration on robot tracked vehicle

Research on Mobile Robot Navigation Method Based on Semantic Information

Vision-based Mobile Robot's Environment Outdoor Perception

Multimodal sensor-based semantic 3D mapping for a large-scale environment