Abstract:3D point cloud classification requires distinct models from 2D image classification due to the divergent characteristics of the respective input data. While 3D point clouds are unstructured and sparse, 2D images are structured and dense. Bridging the domain gap between these two data types is a non-trivial challenge to enable model interchangeability. Recent research using Lattice Point Classifier (LPC) highlights the feasibility of cross-domain applicability. However, the lattice projection operation in LPC generates 2D images with disconnected projected pixels. In this paper, we explore three distinct algorithms for mapping 3D point clouds into 2D images. Through extensive experiments, we thoroughly examine and analyze their performance and defense mechanisms. Leveraging current large foundation models, we scrutinize the feature disparities between regular 2D images and projected 2D images. The proposed approaches demonstrate superior accuracy and robustness against adversarial attacks. The generative model-based mapping algorithms yield regular 2D images, further minimizing the domain gap from regular 2D classification tasks. The source code is available at <a class="link-external link-https" href="https://github.com/KaidongLi/pytorch-LatticePointClassifier.git" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve several key problems in 3D point cloud classification, specifically including: 1. **Domain Gap between 3D Point Clouds and 2D Images**: - 3D point cloud data is unstructured and sparse, while 2D image data is structured and dense. This difference makes it challenging to directly apply 2D image classification models to 3D point clouds. - The paper narrows this domain gap by exploring three different mapping algorithms to convert 3D point clouds into 2D images. 2. **Robustness against Adversarial Attacks**: - Existing deep - learning models are very vulnerable to adversarial attacks, especially in safety - critical applications (such as self - driving). Therefore, it is crucial to improve the classifier's robustness against adversarial attacks. - The paper analyzes the defense mechanisms of different mapping algorithms and demonstrates the resilience of the basic projection and rendering classifiers under adversarial attacks. 3. **Effectiveness of Feature Representation**: - Different mapping algorithms will produce different 2D image feature representations. The paper experimentally evaluates the performance of these algorithms to determine which method can better preserve the feature information of 3D point clouds. - In particular, the paper explores the feature differences between the generated 2D images and regular 2D images and uses pre - trained base models (such as CLIP) for zero - shot classification tests to quantify the domain gap. 4. **Proposing New Mapping Algorithms**: - The paper proposes three new mapping algorithms: basic projection, graph drawing, and rendering classifiers. These algorithms can not only effectively convert 3D point clouds into 2D images but also improve the model's robustness while maintaining classification accuracy. ### Main Contributions - Proposed a set of structured declarative classifiers, using three different mapping algorithms, and verified the performance of these classifiers through a large number of experiments. - Demonstrated the advantages of classifiers based on generative adversarial networks (GAN) in narrowing the domain gap. - Analyzed the gradient propagation mechanism and studied the defense mechanisms of these models, discovered how to weaken declarative defenses, but also proved the resilience of the basic projection and rendering classifiers. Through these efforts, the paper provides a new perspective for 3D point cloud classification and lays the foundation for future related research.

Robust 3D Point Clouds Classification based on Declarative Defenders

Robust Structured Declarative Classifiers for 3D Point Clouds: Defending Adversarial Attacks with Implicit Gradients

Improving Adversarial Robustness of 3D Point Cloud Classification Models

Point Cloud Domain Adaptation Via Masked Local 3D Structure Prediction

Generating 3D Adversarial Point Clouds under the Principle of LiDARs

Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers

Benchmarking and Analyzing Robust Point Cloud Recognition: Bag of Tricks for Defending Adversarial Examples

CAP: Robust Point Cloud Classification Via Semantic and Structural Modeling.

PointGuard: Provably Robust 3D Point Cloud Classification

IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration

Imperceptible Transfer Attack and Defense on 3D Point Cloud Classification

Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Survey

PCLD: Point Cloud Layerwise Diffusion for Adversarial Purification

The Art of Defense: Letting Networks Fool the Attacker

Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

Adversarial Attack and Defense on Point Sets

Point-Voxel Adaptive Feature Abstraction for Robust Point Cloud Classification

Robust Geometry-Dependent Attack for 3D Point Clouds

Using Multi-Scale and Hierarchical Deep Convolutional Features for 3D Semantic Classification of TLS Point Clouds.

Robust 3D Point Cloud Recognition: Enhancing Robustness with GPT-4 and CLIP Integration

On Adversarial Robustness of 3D Point Cloud Classification under Adaptive Attacks