Abstract:This work explores the use of global and local structures of 3D point clouds as a free and powerful supervision signal for representation learning. Local and global patterns of a 3D object are closely related. Although each part of an object is incomplete, the underlying attributes about the object are shared among all parts, which makes reasoning about the whole object from a single part possible. We hypothesize that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects. Based on this hypothesis, we propose a new framework to learn point cloud representations by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape. Moreover, we extend the unsupervised structural representation learning method to more complex 3D scenes. By introducing structural proxies as the intermediate-level representations between local and global ones, we propose a hierarchical reasoning scheme among local parts, structural proxies, and the overall point cloud to learn powerful 3D representations in an unsupervised manner. Extensive experimental results demonstrate that the unsupervised representations can be very competitive alternatives of supervised representations in discriminative power, and exhibit better performance in generalization ability and robustness. Our method establishes the new state-of-the-art of unsupervised/few-shot 3D object classification and part segmentation. We also show our method can serve as a simple yet effective regime for model pre-training on 3D scene segmentation and detection tasks. We expect our observations to offer a new perspective on learning better representations from data structures instead of human annotations for point cloud understanding.

Semi-Supervised Generative Adversarial Model for 3D Recognition

Self-Supervised Global-Local Structure Modeling for Point Cloud Domain Adaptation with Reliable Voted Pseudo Labels

Improving Adversarial Robustness of 3D Point Cloud Classification Models

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

Semi-supervised Three-dimensional Reconstruction Framework with GAN.

Self-supervised single-view 3D point cloud reconstruction through GAN inversion

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning Representations and Generative Models for 3D Point Clouds

Global-Local Bidirectional Reasoning For Unsupervised Representation Learning Of 3d Point Clouds

Semi-Supervised 3d Object Detection Via Adaptive Pseudo-Labeling

PointGLR: Unsupervised Structural Representation Learning of 3D Point Clouds

SemiGMMPoint: Semi-supervised point cloud segmentation based on Gaussian mixture models

3D Point Cloud Recognition Based on a Multi-View Convolutional Neural Network

PPSAN: Perceptual-aware 3D Point Cloud Segmentation Via Adversarial Learning.

Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching

SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network

Semi-supervised 3D shape segmentation with multilevel consistency and part substitution

Unsupervised 3D Reconstruction from a Single Image via Adversarial Learning

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

CPCGAN: A Controllable 3D Point Cloud Generative Adversarial Network with Semantic Label Generating