Abstract:3D contrastive representation learning has exhibited remarkable efficacy across various downstream tasks. However, existing contrastive learning paradigms based on cosine similarity fail to deeply explore the potential intra-modal hierarchical and cross-modal semantic correlations about multi-modal data in Euclidean space. In response, we seek solutions in hyperbolic space and propose a hyperbolic image-and-pointcloud contrastive learning method (HyperIPC). For the intra-modal branch, we rely on the intrinsic geometric structure to explore the hyperbolic embedding representation of point cloud to capture invariant features. For the cross-modal branch, we leverage images to guide the point cloud in establishing strong semantic hierarchical correlations. Empirical experiments underscore the outstanding classification performance of HyperIPC. Notably, HyperIPC enhances object classification results by 2.8% and few-shot classification outcomes by 5.9% on ScanObjectNN compared to the baseline. Furthermore, ablation studies and confirmatory testing validate the rationality of HyperIPC's parameter settings and the effectiveness of its submodules.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that the existing contrastive learning methods based on Euclidean space cannot deeply explore the hierarchical structure within multimodal data and cross - modal semantic associations when processing 3D point - cloud data. Specifically, current methods rely on cosine similarity, which makes it difficult for them to capture the potential semantic hierarchical structure in point - cloud data, especially performing poorly in the relationship between abstract and concrete concepts at different levels. To solve this problem, the authors propose a new method - Hyperbolic Image - and - Pointcloud Contrastive Learning (HyperIPC), which embeds point - cloud data into hyperbolic space for contrastive learning. Through this method, HyperIPC can more effectively capture the semantic hierarchical structure in point - cloud data and, by combining image information, guide point - clouds to establish stronger semantic hierarchical relationships. ### Specific Problems and Solutions 1. **Problem Description**: - Existing contrastive learning methods use cosine similarity in Euclidean space and cannot fully explore the internal hierarchical structure and cross - modal semantic relevance in point - cloud data. - This leads to less - than - ideal performance of the model when processing data with complex hierarchical structures. 2. **Solutions**: - **Introducing Hyperbolic Space**: Hyperbolic space has a negative constant curvature and can better represent tree - like hierarchical structures. Therefore, the authors choose to embed point - cloud data into hyperbolic space to capture its internal semantic hierarchical structure. - **Intra - modal Hyperbolic Contrastive Learning (IMHCL)**: By performing contrastive learning on point - cloud data in hyperbolic space, the representations of point - clouds of the same category are made closer, and those of different categories are made farther apart. - **Cross - modal Hyperbolic Contrastive Learning (CMHCL)**: Use a pre - trained image encoder to extract 2D information in the image and map it into hyperbolic space to guide point - cloud data to establish stronger semantic hierarchical relationships. - **Optimizing Hyperbolic Embedding**: By adjusting the positions of nodes, align the root node to the origin of the hyperbolic space, and optimize the positions of low - level nodes according to hierarchical information, thereby fully utilizing the expansibility of hyperbolic space. ### Experimental Results The experimental results show that HyperIPC has achieved significant performance improvements in multiple downstream tasks. In particular, in the 3D object classification task on the ScanObjectNN dataset, HyperIPC has a 2.8% improvement compared to the baseline method, and a 5.9% improvement in the few - shot classification task. These results indicate that HyperIPC can more effectively capture the semantic hierarchical structure in point - cloud data and improve the generalization ability of the model. ### Summary This paper solves the problem that existing contrastive learning methods cannot fully explore the semantic hierarchical structure when processing 3D point - cloud data by introducing hyperbolic space and combining image information. The experimental results verify the effectiveness and superiority of this method.

Hyperbolic Image-and-Pointcloud Contrastive Learning for 3D Classification