3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion

Yixuan Li,Weidong Yang,Ben Fei

2024-04-10

Abstract:Point cloud completion aims to generate a complete and high-fidelity point cloud from an initially incomplete and low-quality input. A prevalent strategy involves leveraging Transformer-based models to encode global features and facilitate the reconstruction process. However, the adoption of pooling operations to obtain global feature representations often results in the loss of local details within the point cloud. Moreover, the attention mechanism inherent in Transformers introduces additional computational complexity, rendering it challenging to handle long sequences effectively. To address these issues, we propose 3DMambaComplete, a point cloud completion network built on the novel Mamba framework. It comprises three modules: HyperPoint Generation encodes point cloud features using Mamba's selection mechanism and predicts a set of Hyperpoints. A specific offset is estimated, and the down-sampled points become HyperPoints. The HyperPoint Spread module disperses these HyperPoints across different spatial locations to avoid concentration. Finally, a deformation method transforms the 2D mesh representation of HyperPoints into a fine-grained 3D structure for point cloud reconstruction. Extensive experiments conducted on various established benchmarks demonstrate that 3DMambaComplete surpasses state-of-the-art point cloud completion methods, as confirmed by qualitative and quantitative analyses.

Computer Vision and Pattern Recognition,Graphics

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve

This paper aims to address the key challenges in point cloud completion. Point cloud completion involves generating a complete and high-fidelity point cloud by recovering the missing parts from an initially incomplete and low-quality input. Current methods primarily rely on Transformer-based models to encode global features and facilitate the reconstruction process. However, these methods have two main issues:

1. **Loss of Local Details**: During the encoding process, local details in the point cloud are often lost when obtaining global feature representations through pooling operations.
2. **High Computational Complexity**: The attention mechanism in Transformer models introduces additional computational complexity, making it difficult to efficiently handle long sequence data.

To address these issues, the authors propose 3DMambaComplete, a point cloud completion network based on the novel Mamba framework. 3DMambaComplete achieves point cloud completion through the following three modules:

1. **HyperPoint Generation**: Utilizes Mamba's selection mechanism to encode point cloud features and predict a set of Hyperpoints. By estimating specific offsets, the downsampled points become Hyperpoints.
2. **HyperPoint Spread**: Distributes these Hyperpoints to different spatial locations to avoid concentration.
3. **Point Deformation**: Deforms the 2D grid representation of Hyperpoints into a fine-grained 3D structure for point cloud reconstruction.

Through extensive experiments, 3DMambaComplete demonstrates outstanding performance on multiple benchmarks, surpassing existing point cloud completion methods. These experimental results are validated through both qualitative and quantitative analyses.