Abstract:Point cloud segmentation is essential for scene understanding, which provides advanced information for many applications, such as autonomous driving, robots, and virtual reality. To improve the accuracy and robustness of point cloud segmentation, many researchers have attempted to fuse camera images to complement the color and texture information. The common fusion strategy is the combination of convolutional operations with concatenation, element-wise addition or element-wise multiplication. However, conventional convolutional operators tend to confine the fusion of modal features within their receptive fields, which can be incomplete and limited. In addition, the inability of encoder-decoder segmentation networks to explicitly perceive segmentation boundary information results in semantic ambiguity and classification errors at object edges. These errors are further amplified in point cloud segmentation tasks, significantly affecting the accuracy of point cloud segmentation. To address the above issues, we propose a novel self-attention multi-modal fusion semantic segmentation network for point cloud semantic segmentation. Firstly, to effectively fuse different modal features, we propose a Self-Cross Fusion Module (SCF), which models long-range modality dependencies and transfers complementary image information to the point cloud to fully leverage the modality-specific advantages. Secondly, we design the Salience Refinement Module (SR), which calculates the importance of channels in the feature maps and global descriptors to enhance the representation capability of salient modal features. Finally, we propose the Local-aware Anisotropy Loss measure the element-level importance in the data and explicitly provide boundary information for the model, which alleviates the inherent semantic ambiguity problem in segmentation networks. Extensive experiments on two benchmark datasets demonstrate that our proposed method surpasses current state-of-the-art methods.

Enhanced Local Feature Learning with Simple Offset Attention for Semantic Segmentation of Large-scale Point Clouds

LFEA-Net: semantic segmentation for urban point cloud scene via local feature extraction and aggregation

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Local Transformer Network on 3D Point Cloud Semantic Segmentation

Efficiently Expanding Receptive Fields: Local Split Attention and Parallel Aggregation for Enhanced Large-scale Point Cloud Semantic Segmentation

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

Large-scale point cloud semantic segmentation via local perception and global descriptor vector

A LiDAR Point Cloud Semantic Segmentation Algorithm Based on Attention Mechanism and Hybrid CNN-LSTM

ALS Point Cloud Semantic Segmentation Based on Graph Convolution and Transformer with Elevation Attention

LLGF-Net: Learning Local and Global Feature Fusion for 3D Point Cloud Semantic Segmentation

A Large-Scale Network Construction and Lightweighting Method for Point Cloud Semantic Segmentation

Mining Local Geometric Structure for Large-Scale 3D Point Clouds Semantic Segmentation

LPFE-Net: a local parallel feature extraction network for large-scale point cloud semantic segmentation

Point Attention Network for Semantic Segmentation of 3D Point Clouds

Sampling-attention Deep Learning Network with Transfer Learning for Large-Scale Urban Point Cloud Semantic Segmentation

Multi-modal LiDAR Point Cloud Semantic Segmentation with Salience Refinement and Boundary Perception

A Local-Global Feature Fusing Method for Point Clouds Semantic Segmentation

Semantic Segmentation of Point Cloud Scene via Multi-Scale Feature Aggregation and Adaptive Fusion

DLA-Net: Learning Dual Local Attention Features for Semantic Segmentation of Large-Scale Building Facade Point Clouds

LSLPCT: an Enhanced Local Semantic Learning Transformer for 3-D Point Cloud Analysis

Semantic Segmentation Method for Outdoor Point Clouds Based on Contextual Attention