Abstract:Training deep models for LiDAR semantic segmentation is challenging due to the inherent sparsity of point clouds. Utilizing temporal data is a natural remedy against the sparsity problem as it makes the input signal denser. However, previous multi-frame fusion algorithms fall short in utilizing sufficient temporal information due to the memory constraint, and they also ignore the informative temporal images. To fully exploit rich information hidden in long-term temporal point clouds and images, we present the Temporal Aggregation Network, termed TASeg. Specifically, we propose a Temporal LiDAR Aggregation and Distillation (TLAD) algorithm, which leverages historical priors to assign different aggregation steps for different classes. It can largely reduce memory and time overhead while achieving higher accuracy. Besides, TLAD trains a teacher injected with gt priors to distill the model, further boosting the performance. To make full use of temporal images, we design a Temporal Image Aggregation and Fusion (TIAF) module, which can greatly expand the camera FOV and enhance the present features. Temporal LiDAR points in the camera FOV are used as mediums to transform temporal image features to the present coordinate for temporal multi-modal fusion. Moreover, we develop a Static-Moving Switch Augmentation (SMSA) algorithm, which utilizes sufficient temporal information to enable objects to switch their motion states freely, thus greatly increasing static and moving training samples. Our TASeg ranks 1st on three challenging tracks, i.e., SemanticKITTI single-scan track, multi-scan track and nuScenes LiDAR segmentation track, strongly demonstrating the superiority of our method. Codes are available at <a class="link-external link-https" href="https://github.com/LittlePey/TASeg" rel="external noopener nofollow">this https URL</a>.

MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory

LiDAR Video Object Segmentation with Dynamic Kernel Refinement

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Memory-based Semantic Segmentation for Off-road Unstructured Natural Environments

Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery

Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving

Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study

SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation

Fast Context-Awareness Encoder for LiDAR Point Semantic Segmentation

TeFF: Tracking-enhanced Forgetting-free Few-shot 3D LiDAR Semantic Segmentation

TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation

Human Segmentation with Dynamic LiDAR Data

Temporal Feature Matching and Propagation for Semantic Segmentation on 3D Point Cloud Sequences

StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory

Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

Hybrid CNN-LSTM Architecture for LiDAR Point Clouds Semantic Segmentation

Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

TempNet: Online Semantic Segmentation on Large-scale Point Cloud Series

TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

Learning Spatial and Temporal Variations for 4D Point Cloud Segmentation