Abstract:Given the prominence of 3-D sensors in recent years, 3-D point clouds are worthy to be further investigated for environment perception and scene understanding. Learning accurate local and global contexts in point clouds is pivotal for semantic segmentation, and neighbor aggregation (NA) and transformers have achieved notable success in local and global perception in point cloud analysis, respectively. Nevertheless, studying each independently is far from the optimal solution for comprehensive feature learning. To address this, we take a novel step toward investigating and integrating the structures of NA and transformers. In this article, we introduce Point Neighbor Aggregation with Transformer (PointNAT), a conceptually straightforward and effective approach aiming to enhance the performance of 3-D point cloud semantic segmentation. PointNAT consists of an NA block (NAB) for local perception, a point transformer block (PTB) for global modeling, and a hybrid block to connect NABs and PTBs. NABs effectively learn complex local features at varying scales through an improved NA operation and a multihead mechanism. PTBs efficiently perform global attention using a small set of learnable key points. Hybrid blocks serve as high-and-low frequency signal hybridizers, merging the strengths of these two blocks by adaptively assigning hybrid weights to local and global contexts. We have evaluated the performance of PointNAT with state-of-the-art networks on several benchmarks, including Stanford Large-Scale 3-D Indoor Spaces (S3DIS), Toronto3D, and SensatUrban. PointNAT achieves mean intersection over union (mIoU) scores of 77.8%, 84.7%, and 65.2% in these three datasets. Furthermore, it outperforms the baseline approach PointNeXt by 3.0%, 1.3%, and 4.2% while utilizing only 59.9% of the parameters and 15.2% of the floating-point operations (FLOPs). The results demonstrate PointNAT's superior ability in accurately segmenting large-scale 3-D point cloud scenes, emphasizing its potential to advance environment perception and scene understanding. Our code is available at https://github.com/zeng-ziyin/PointNAT.

NDT-Transformer: Large-Scale 3D Point Cloud Localisation using the Normal Distribution Transform Representation

NDT-Transformer: Large-Scale 3D Point Cloud Localisation using the Normal Distribution Transform Representation

Geo-Localization with Transformer-Based 2D-3D Match Network

Deep Place Recognition for 3D LiDAR Point Cloud with Transformer

PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer

An Efficient 3-D Point Cloud Place Recognition Approach Based on Feature Point Extraction and Transformer

TopSPR-Net: Topology Aware Segment-Level Point Cloud Learning Descriptors for Three-Dimensional Place Recognition in Large-Scale Environments

TransLoc3D : Point Cloud based Large-scale Place Recognition using Adaptive Receptive Fields

Pyramid Point Cloud Transformer for Large-Scale Place Recognition.

CVTNet: A Cross-View Transformer Network for LiDAR-Based Place Recognition in Autonomous Driving Environments

D2T-Net: A dual-domain transformer network exploiting spatial and channel dimensions for semantic segmentation of urban mobile laser scanning point clouds

CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data

LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification

Collect-and-Distribute Transformer for 3D Point Cloud Analysis

Spatial Transformer for 3D Point Clouds

BPT: Binary Point Cloud Transformer for Place Recognition

Stratified Transformer for 3D Point Cloud Segmentation

HiTPR: Hierarchical Transformer for Place Recognition in Point Cloud

TBFNT3D: Two-Branch Fusion Network with Transformer for Multimodal Indoor 3D Object Detection