Abstract:Semantic Scene Completion (SSC) aims to jointly generate space occupancies and semantic labels for complex 3D scenes. Most existing SSC models focus on volumetric representations, which are memory-inefficient for large outdoor spaces. Point clouds provide a lightweight alternative but existing benchmarks lack outdoor point cloud scenes with semantic labels. To address this, we introduce PointSSC, the first cooperative vehicle-infrastructure point cloud benchmark for semantic scene completion. These scenes exhibit long-range perception and minimal occlusion. We develop an automated annotation pipeline leveraging Semantic Segment Anything to efficiently assign semantics. To benchmark progress, we propose a LiDAR-based model with a Spatial-Aware Transformer for global and local feature extraction and a Completion and Segmentation Cooperative Module for joint completion and segmentation. PointSSC provides a challenging testbed to drive advances in semantic point cloud completion for real-world navigation. The code and datasets are available at <a class="link-external link-https" href="https://github.com/yyxssm/PointSSC" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the deficiencies of existing Semantic Scene Completion (SSC) models when dealing with large - scale outdoor scenes. Specifically, the existing SSC models mainly focus on volume representation, which is inefficient and has high memory consumption when dealing with large outdoor spaces. In addition, the existing benchmarks lack outdoor point - cloud scenes with semantic labels. Therefore, this paper proposes PointSSC, which is the first collaborative vehicle - infrastructure point - cloud benchmark for semantic scene completion, aiming to solve the following problems: 1. **Limitations of the dataset**: Most of the existing SSC datasets rely on data collected by on - board sensors. These sensors have a limited sensing range and are easily affected by occlusion. For example, SemanticKITTI only provides semantic scenes from the front - view perspective, while SurroundOcc and OpenOccuPancy, although combined with surrounding perspectives, still cannot effectively handle occluded areas. Occ3D uses ray - casting to generate occlusion masks, but only for improving evaluation metrics, not for improving the quality of ground - truth labels. 2. **Complexity of outdoor scenes**: The existing vehicle - view - based datasets cannot capture long - distance perception and ubiquitous occlusion phenomena common in real - driving environments. Therefore, a dataset that can obtain data from an infrastructure perspective is needed to provide more abundant and complete semantic annotations. 3. **Effectiveness of the model**: To verify the effectiveness of the PointSSC dataset, this paper proposes a LiDAR - based model that contains a Spatial - Aware Transformer and a Completion and Segmentation Cooperative Module (CSCM) for jointly performing completion and segmentation tasks. By solving these problems, PointSSC aims to provide a challenging test platform for semantic point - cloud completion research in outdoor autonomous navigation and promote the further development of this field.

PointSSC: A Cooperative Vehicle-Infrastructure Point Cloud Benchmark for Semantic Scene Completion

SSC: Semantic Scan Context for Large-Scale Place Recognition

SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving

SSCBench: Monocular 3D Semantic Scene Completion Benchmark in Street Views

V2VSSC: A 3D Semantic Scene Completion Benchmark for Perception with Vehicle to Vehicle Communication

2D Semantic-Guided Semantic Scene Completion

Semantic Segmentation-assisted Scene Completion for LiDAR Point Clouds

Voxel- and Bird's-Eye-View-Based Semantic Scene Completion for LiDAR Point Clouds

Point Cloud Semantic Scene Completion from RGB-D Images.

Semantic scene completion with point cloud representation and transformer-based feature fusion

CSPC-Dataset: New LiDAR Point Cloud Dataset and Benchmark for Large-Scale Scene Semantic Segmentation

LiDAR-Camera Continuous Fusion in Voxelized Grid for Semantic Scene Completion

SCPNet: Semantic Scene Completion on Point Cloud

Camera-based 3D Semantic Scene Completion with Sparse Guidance Network

Label-efficient Semantic Scene Completion with Scribble Annotations

Semantic Point Completion Network for 3D Semantic Scene Completion.

DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models

DepthSSC: Depth-Spatial Alignment and Dynamic Voxel Resolution for Monocular 3D Semantic Scene Completion

Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion

Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges