Abstract:Point cloud segmentation is essential for scene understanding, which provides advanced information for many applications, such as autonomous driving, robots, and virtual reality. To improve the accuracy and robustness of point cloud segmentation, many researchers have attempted to fuse camera images to complement the color and texture information. The common fusion strategy is the combination of convolutional operations with concatenation, element-wise addition or element-wise multiplication. However, conventional convolutional operators tend to confine the fusion of modal features within their receptive fields, which can be incomplete and limited. In addition, the inability of encoder-decoder segmentation networks to explicitly perceive segmentation boundary information results in semantic ambiguity and classification errors at object edges. These errors are further amplified in point cloud segmentation tasks, significantly affecting the accuracy of point cloud segmentation. To address the above issues, we propose a novel self-attention multi-modal fusion semantic segmentation network for point cloud semantic segmentation. Firstly, to effectively fuse different modal features, we propose a Self-Cross Fusion Module (SCF), which models long-range modality dependencies and transfers complementary image information to the point cloud to fully leverage the modality-specific advantages. Secondly, we design the Salience Refinement Module (SR), which calculates the importance of channels in the feature maps and global descriptors to enhance the representation capability of salient modal features. Finally, we propose the Local-aware Anisotropy Loss measure the element-level importance in the data and explicitly provide boundary information for the model, which alleviates the inherent semantic ambiguity problem in segmentation networks. Extensive experiments on two benchmark datasets demonstrate that our proposed method surpasses current state-of-the-art methods.

Fine-Grained Semantic Information Preservation and Misclassification-Aware Loss for 3D Point Cloud

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

Multi-modal LiDAR Point Cloud Semantic Segmentation with Salience Refinement and Boundary Perception

Backward Attentive Fusing Network With Local Aggregation Classifier for 3D Point Cloud Semantic Segmentation

Semantic Context Encoding for Accurate 3D Point Cloud Segmentation

An attention-based bilateral feature fusion network for 3D point cloud

Semantic-aware Transmission for Robust Point Cloud Classification

A feature fusion-based attention graph convolutional network for 3D classification and segmentation

Point Attention Network for Semantic Segmentation of 3D Point Clouds

Using Multi-Scale and Hierarchical Deep Convolutional Features for 3D Semantic Classification of TLS Point Clouds.

pCTFusion: Point Convolution-Transformer Fusion with Semantic Aware Loss for Outdoor LiDAR Point Cloud Segmentation

Multi-scale Network with Attentional Multi-resolution Fusion for Point Cloud Semantic Segmentation

Background-Aware 3D Point Cloud Segmentationwith Dynamic Point Feature Aggregation

3D Graph Embedding Learning with a Structure-aware Loss Function for Point Cloud Semantic Instance Segmentation

Semantic Segmentation of Point Cloud Scene via Multi-Scale Feature Aggregation and Adaptive Fusion

PointMM: Point Cloud Semantic Segmentation CNN under Multi-Spatial Feature Encoding and Multi-Head Attention Pooling

Towards Deeper and Better Multi-view Feature Fusion for 3D Semantic Segmentation

SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation

MSIDA-Net: Point Cloud Semantic Segmentation via Multi-Spatial Information and Dual Adaptive Blocks

Robust 3D Semantic Segmentation Method Based on Multi-Modal Collaborative Learning

Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation