Abstract:Given the prominence of 3-D sensors in recent years, 3-D point clouds are worthy to be further investigated for environment perception and scene understanding. Learning accurate local and global contexts in point clouds is pivotal for semantic segmentation, and neighbor aggregation (NA) and transformers have achieved notable success in local and global perception in point cloud analysis, respectively. Nevertheless, studying each independently is far from the optimal solution for comprehensive feature learning. To address this, we take a novel step toward investigating and integrating the structures of NA and transformers. In this article, we introduce Point Neighbor Aggregation with Transformer (PointNAT), a conceptually straightforward and effective approach aiming to enhance the performance of 3-D point cloud semantic segmentation. PointNAT consists of an NA block (NAB) for local perception, a point transformer block (PTB) for global modeling, and a hybrid block to connect NABs and PTBs. NABs effectively learn complex local features at varying scales through an improved NA operation and a multihead mechanism. PTBs efficiently perform global attention using a small set of learnable key points. Hybrid blocks serve as high-and-low frequency signal hybridizers, merging the strengths of these two blocks by adaptively assigning hybrid weights to local and global contexts. We have evaluated the performance of PointNAT with state-of-the-art networks on several benchmarks, including Stanford Large-Scale 3-D Indoor Spaces (S3DIS), Toronto3D, and SensatUrban. PointNAT achieves mean intersection over union (mIoU) scores of 77.8%, 84.7%, and 65.2% in these three datasets. Furthermore, it outperforms the baseline approach PointNeXt by 3.0%, 1.3%, and 4.2% while utilizing only 59.9% of the parameters and 15.2% of the floating-point operations (FLOPs). The results demonstrate PointNAT's superior ability in accurately segmenting large-scale 3-D point cloud scenes, emphasizing its potential to advance environment perception and scene understanding. Our code is available at https://github.com/zeng-ziyin/PointNAT.

Point Transformer-Based Salient Object Detection Network for 3-D Measurement Point Clouds

PSFormer: Point Transformer for 3D Salient Object Detection

PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer

CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer

Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection

PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection

Improving 3D Object Detection with Channel-wise Transformer

A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection

Weakly Supervised Point Clouds Transformer for 3D Object Detection

Boosting Salient Object Detection with Transformer-based Asymmetric Bilateral U-Net

Transformer with large convolution kernel decoder network for salient object detection in optical remote sensing images

Hierarchical Point Attention for Indoor 3D Object Detection

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

Salient Object Detection for Point Clouds

Long-short Range Adaptive Transformer with Dynamic Sampling for 3D Object Detection

3D Landmark Detection on Human Point Clouds: A Benchmark and A Dual Cascade Point Transformer Framework

Dual-Stream Feature Collaboration Perception Network for Salient Object Detection in Remote Sensing Images

3DPCTN: Two 3D Local-Object Point-Cloud-Completion Transformer Networks Based on Self-Attention and Multi-Resolution

TANet: Transformer-based Asymmetric Network for RGB-D Salient Object Detection

DOCTR: Disentangled Object-Centric Transformer for Point Scene Understanding

Collect-and-Distribute Transformer for 3D Point Cloud Analysis