Abstract:Point cloud segmentation, which helps us understand the environment of specific structures and objects, can be performed in class-specific and class-agnostic ways. We propose a novel region-based transformer model called Region-Transformer for performing class-agnostic point cloud segmentation. The model utilizes a region-growth approach and self-attention mechanism to iteratively expand or contract a region by adding or removing points. It is trained on simulated point clouds with instance labels only, avoiding semantic labels. Attention-based networks have succeeded in many previous methods of performing point cloud segmentation. However, a region-growth approach with attention-based networks has yet to be used to explore its performance gain. To our knowledge, we are the first to use a self-attention mechanism in a region-growth approach. With the introduction of self-attention to region-growth that can utilize local contextual information of neighborhood points, our experiments demonstrate that the Region-Transformer model outperforms previous class-agnostic and class-specific methods on indoor datasets regarding clustering metrics. The model generalizes well to large-scale scenes. Key advantages include capturing long-range dependencies through self-attention, avoiding the need for semantic labels during training, and applicability to a variable number of objects. The Region-Transformer model represents a promising approach for flexible point cloud segmentation with applications in robotics, digital twinning, and autonomous vehicles.

What problem does this paper attempt to address?

The paper attempts to address the problem of class-agnostic segmentation in point cloud data. Specifically, most existing point cloud segmentation methods rely heavily on semantic labels, meaning they need to know in advance which category each point belongs to, which limits the flexibility and applicability of these methods. In contrast, class-agnostic segmentation methods do not require such prior knowledge, allowing for more flexible handling of point cloud data in different environments. To achieve this goal, the authors propose a new model based on region growing and self-attention mechanisms—Region-Transformer. By combining self-attention mechanisms with region growing methods, this model can effectively segment point cloud data without semantic labels. Specifically, the main contributions of the model include: 1. **Utilizing self-attention mechanisms**: By applying self-attention mechanisms within local neighborhoods, the model can capture fine-grained relationships between points and their neighboring points, thereby determining segmentation boundaries more precisely. 2. **Region growing method**: The model adopts an iterative approach, gradually expanding or contracting regions by adding or removing points to complete the segmentation task. 3. **No need for semantic labels**: The model does not require semantic labels during training, only instance labels, which gives the model higher flexibility and allows it to handle any number of objects. 4. **Suitable for large-scale scenes**: The model is not only applicable to small-scale data but also handles large-scale scenes well, such as factories and large buildings. Experimental results show that Region-Transformer significantly outperforms existing class-specific and class-agnostic segmentation methods on both indoor and outdoor datasets, particularly excelling in clustering metrics. Additionally, the model demonstrates excellent computational efficiency, achieving fast inference speeds while maintaining high accuracy. These characteristics make it widely applicable in fields such as robotic perception, autonomous navigation, digital twins, and augmented reality.

Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation

Local Transformer Network on 3D Point Cloud Semantic Segmentation

3DPCT: 3D Point Cloud Transformer with Dual Self-attention

CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning

PointCAT: Cross-Attention Transformer for point cloud

Attention-based Transformation from Latent Features to Point Clouds

Spatial Transformer for 3D Point Clouds

Stratified Transformer for 3D Point Cloud Segmentation

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

SGT-Net: A Transformer-Based Stratified Graph Convolutional Network for 3D Point Cloud Semantic Segmentation

Point Cloud Understanding via Attention-Driven Contrastive Learning

OctFormer: Octree-based Transformers for 3D Point Clouds

Collect-and-Distribute Transformer for 3D Point Cloud Analysis

Dynamic clustering transformer network for point cloud segmentation

Modify Self-Attention Via Skeleton Decomposition for Effective Point Cloud Transformer.

PointMT: Efficient Point Cloud Analysis with Hybrid MLP-Transformer Architecture

Transformer based 3D tooth segmentation via point cloud region partition

Regional-to-Local Point-Voxel Transformer for Large-Scale Indoor 3D Point Cloud Semantic Segmentation

MPCT: Multiscale Point Cloud Transformer with a Residual Network