Hierarchical Point Attention for Indoor 3D Object Detection

Manli Shu,Le Xue,Ning Yu,Roberto Martín-Martín,Caiming Xiong,Tom Goldstein,Juan Carlos Niebles,Ran Xu

2024-05-09

Abstract:3D object detection is an essential vision technique for various robotic systems, such as augmented reality and domestic robots. Transformers as versatile network architectures have recently seen great success in 3D point cloud object detection. However, the lack of hierarchy in a plain transformer restrains its ability to learn features at different scales. Such limitation makes transformer detectors perform worse on smaller objects and affects their reliability in indoor environments where small objects are the majority. This work proposes two novel attention operations as generic hierarchical designs for point-based transformer detectors. First, we propose Aggregated Multi-Scale Attention (MS-A) that builds multi-scale tokens from a single-scale input feature to enable more fine-grained feature learning. Second, we propose Size-Adaptive Local Attention (Local-A) with adaptive attention regions for localized feature aggregation within bounding box proposals. Both attention operations are model-agnostic network modules that can be plugged into existing point cloud transformers for end-to-end training. We evaluate our method on two widely used indoor detection benchmarks. By plugging our proposed modules into the state-of-the-art transformer-based 3D detectors, we improve the previous best results on both benchmarks, with more significant improvements on smaller objects.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of performance imbalance in existing Transformer-based detectors for indoor 3D object detection, particularly the poor detection performance for small objects. Specifically, due to the sparsity of point cloud data and the lack of hierarchical structure in existing Transformer models, these models perform poorly when handling small objects, affecting their reliability and robustness in indoor environments.

To tackle this challenge, the authors propose two new attention mechanism modules—Aggregated Multi-Scale Attention (MS-A) and Size-Adaptive Local Attention (Local-A)—aiming to improve the detection performance for small objects by introducing multi-level and local feature learning. These two modules can be seamlessly integrated into existing point cloud Transformer models to enhance their overall performance in 3D object detection tasks, especially in detecting small objects.

By evaluating on two widely used indoor 3D detection benchmark datasets (ScanNetV2 and SUN RGB-D), the authors demonstrate that their method not only significantly improves the detection accuracy for small objects but also enhances the overall performance of existing state-of-the-art models without adding excessive parameters.