Abstract:Given the prominence of 3-D sensors in recent years, 3-D point clouds are worthy to be further investigated for environment perception and scene understanding. Learning accurate local and global contexts in point clouds is pivotal for semantic segmentation, and neighbor aggregation (NA) and transformers have achieved notable success in local and global perception in point cloud analysis, respectively. Nevertheless, studying each independently is far from the optimal solution for comprehensive feature learning. To address this, we take a novel step toward investigating and integrating the structures of NA and transformers. In this article, we introduce Point Neighbor Aggregation with Transformer (PointNAT), a conceptually straightforward and effective approach aiming to enhance the performance of 3-D point cloud semantic segmentation. PointNAT consists of an NA block (NAB) for local perception, a point transformer block (PTB) for global modeling, and a hybrid block to connect NABs and PTBs. NABs effectively learn complex local features at varying scales through an improved NA operation and a multihead mechanism. PTBs efficiently perform global attention using a small set of learnable key points. Hybrid blocks serve as high-and-low frequency signal hybridizers, merging the strengths of these two blocks by adaptively assigning hybrid weights to local and global contexts. We have evaluated the performance of PointNAT with state-of-the-art networks on several benchmarks, including Stanford Large-Scale 3-D Indoor Spaces (S3DIS), Toronto3D, and SensatUrban. PointNAT achieves mean intersection over union (mIoU) scores of 77.8%, 84.7%, and 65.2% in these three datasets. Furthermore, it outperforms the baseline approach PointNeXt by 3.0%, 1.3%, and 4.2% while utilizing only 59.9% of the parameters and 15.2% of the floating-point operations (FLOPs). The results demonstrate PointNAT's superior ability in accurately segmenting large-scale 3-D point cloud scenes, emphasizing its potential to advance environment perception and scene understanding. Our code is available at https://github.com/zeng-ziyin/PointNAT.

EP-Net: Improving Point Cloud Learning Efficiency Through Feature Decoupling

PointeNet: A Lightweight Framework for Effective and Efficient Point Cloud Analysis

Dual-scale Point Cloud Completion Network Based on High-Frequency Feature Fusion

APP-Net: Auxiliary-Point-Based Push and Pull Operations for Efficient Point Cloud Recognition

PointNetX: Part Segmentation Based on PointNet Promotion

Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis.

BiPointNet: Binary Neural Network for Point Clouds

PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies

NeiEA-NET: Semantic segmentation of large-scale point cloud scene via neighbor enhancement and aggregation

DCNet: Large-Scale Point Cloud Semantic Segmentation with Discriminative and Efficient Feature Aggregation

Efficiently Expanding Receptive Fields: Local Split Attention and Parallel Aggregation for Enhanced Large-scale Point Cloud Semantic Segmentation

On-the-fly Point Feature Representation for Point Clouds Analysis

High-Performance Feature Extraction Network for Point Cloud Semantic Segmentation

A Large-Scale Network Construction and Lightweighting Method for Point Cloud Semantic Segmentation

PointNAC: Copula-Based Point Cloud Semantic Segmentation Network

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

PeP: a Point enhanced Painting method for unified point cloud tasks

Point Deformable Network with Enhanced Normal Embedding for Point Cloud Analysis

Aggregation and Purification: Dual Enhancement Network for Point Cloud Few-shot Segmentation

Point Attention Network for Point Cloud Semantic Segmentation.

Semantic Segmentation of Aerial Laser Point Clouds Based on Deep-Residual Enhanced Coding of Multi-Feature Information