Abstract:The 3D semantic map plays an increasingly important role in a wide variety of applications, especially for many kinds of task-driven robots. In this paper, we present a semantic mapping methodology for 3D semantic map obtaining from RGB-D scans. In contrast to existing methods that use 3D annotated information as supervisory, we focus on accurate 2D frame labeling and combine labels in 3D space using semantic fusion mechanism. For scene parsing, a two-stream network with a novel discriminatory mask loss is proposed to explore sufficient extraction and fusion of RGB and depth information achieving steadily semantic segmentation. The discriminatory mask guides the cross-entropy loss function and interprets the influence of different pixels on back-propagation, which reduces the harmful effects of the depth noise or the fallible annotation at the edges of objects. After the correspondences between frames are provided, these semantic frames are fused in unified 3D coordinates using the novel label-oriented voxelgrid filter. It can ensure the intra-frame spatial continuity and the inter-frame spatiotemporal consistency through introducing the label-oriented statistical principle into labeled point clouds. In order to avoid the unfavorable interference between uncorrelated frames, we further propose an adaptive grouping algorithm by applying the view frustum filter to group frames with sufficient overlap as a segment. To this end, we demonstrate the effectiveness of the proposed method on the 2D/3D semantic label benchmark of ScanNetv2 and Cityscapes datasets.

Dense Spatial Segmentation from Sparse Semantic Information

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

Object-aware Semantic Mapping of Indoor Scenes Using Octomap

A Unified Semantics Space Model

A NEW REAL-TIME 3D DENSE SEMANTIC MAPPING SYSTEM FOR LARGE-SCALE ENVIRONMENTS

A NEW REAL-TIME 3D DENSE SEMANTIC MAPPING SYSTEM FOR LARGE-SCALE ENVIRONMENTS, 12-23.

Semantic Reconstruction based on RGB Image and Sparse Depth

Towards Robotic Semantic Segmentation of Supporting Surfaces

Semantic Dense Reconstruction with Consistent Scene Segments

Generating Spatial Semantic Representations for Indoor Global Mapping

3D Segmentation Learning from Sparse Annotations and Hierarchical Descriptors

An Approach for Construct Semantic Map with Scene Classification and Object Semantic Segmentation

OccuSeg: Occupancy-Aware 3D Instance Segmentation

Explicit3D: Graph Network with Spatial Inference for Single Image 3D Object Detection

Spatial Pyramid Based Graph Reasoning for Semantic Segmentation

Geometrically consistent plane extraction for dense indoor 3D maps segmentation

Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding

RGB-D Semantic Segmentation and Label-Oriented Voxelgrid Fusion for Accurate 3D Semantic Mapping

Online Scene Semantic Understanding Based on Sparsely Correlated Network for AR

Mapping High-level Semantic Regions in Indoor Environments without Object Recognition

Real-time Semantic 3D Dense Occupancy Mapping with Efficient Free Space Representations