Training-Free Point Cloud Recognition Based on Geometric and Semantic Information Fusion

Yan Chen,Di Huang,Zhichao Liao,Xi Cheng,Xinghui Li,Lone Zeng

2024-09-11

Abstract:The trend of employing training-free methods for point cloud recognition is becoming increasingly popular due to its significant reduction in computational resources and time costs. However, existing approaches are limited as they typically extract either geometric or semantic features. To address this limitation, we are the first to propose a novel training-free method that integrates both geometric and semantic features. For the geometric branch, we adopt a non-parametric strategy to extract geometric features. In the semantic branch, we leverage a model aligned with text features to obtain semantic features. Additionally, we introduce the GFE module to complement the geometric information of point clouds and the MFF module to improve performance in few-shot settings. Experimental results demonstrate that our method outperforms existing state-of-the-art training-free approaches on mainstream benchmark datasets, including ModelNet and ScanObiectNN.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in point cloud recognition tasks, existing training - free methods usually only extract geometric features or semantic features, resulting in an incomplete understanding of point cloud data by the model. To overcome this limitation, this paper proposes a new training - free method, which fuses geometric features and semantic features for the first time to improve the performance of point cloud recognition.

Specifically, the paper aims to:

1. **Integrate geometric and semantic information**: By combining geometric features and semantic features, the model can understand and represent point cloud data more comprehensively.
2. **Reduce computational resources and time costs**: Utilize training - free methods to avoid the problem of requiring a large amount of computational resources and time for model training in traditional methods.
3. **Improve few - sample learning performance**: Introduce the Memory Feature Filtering (MFF) module and the Geometric Feature Enhancement (GFE) module to enhance the performance of the model in few - sample classification tasks.

### Method overview

- **Geometric feature extraction**:
- Use non - parametric strategies (such as farthest point sampling (FPS), k - nearest neighbors (k - NN), and pooling operations) to extract geometric features from point clouds.
- Introduce the GFE module to enhance geometric information, including converting Cartesian coordinates to spherical coordinate systems, and combining edge vectors and length information.

- **Semantic feature extraction**:
- Use a pre - trained ULIP 3D encoder to directly extract semantic features from point clouds. This encoder has been aligned with natural language descriptions during the pre - training stage.

- **Feature fusion**:
- Fuse geometric features and semantic features by weighted summation. The formula is as follows:
\[
f_{\text{fuse}}=\alpha\cdot f_{\text{geo}}+(1 - \alpha)\cdot f_{\text{sem}}
\]
- Where \( f_{\text{geo}} \) is the geometric feature, \( f_{\text{sem}} \) is the semantic feature, and \( \alpha \) is a hyperparameter.

- **Few - sample learning optimization**:
- Use the K - Means++ clustering algorithm to select the most representative samples to reduce redundancy and retain key features.
- Propose the MFF module to further improve the performance of few - sample learning tasks.

### Experimental results

The experimental results show that this method significantly outperforms existing training - free methods on multiple benchmark datasets (such as ModelNet and ScanObjectNN), proving its effectiveness and superiority in point cloud recognition tasks.

Through these improvements, this paper successfully solves the limitations of existing training - free methods in point cloud recognition and provides new ideas for future research.