Abstract:This work explores the use of global and local structures of 3D point clouds as a free and powerful supervision signal for representation learning. Local and global patterns of a 3D object are closely related. Although each part of an object is incomplete, the underlying attributes about the object are shared among all parts, which makes reasoning about the whole object from a single part possible. We hypothesize that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects. Based on this hypothesis, we propose a new framework to learn point cloud representations by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape. Moreover, we extend the unsupervised structural representation learning method to more complex 3D scenes. By introducing structural proxies as the intermediate-level representations between local and global ones, we propose a hierarchical reasoning scheme among local parts, structural proxies, and the overall point cloud to learn powerful 3D representations in an unsupervised manner. Extensive experimental results demonstrate that the unsupervised representations can be very competitive alternatives of supervised representations in discriminative power, and exhibit better performance in generalization ability and robustness. Our method establishes the new state-of-the-art of unsupervised/few-shot 3D object classification and part segmentation. We also show our method can serve as a simple yet effective regime for model pre-training on 3D scene segmentation and detection tasks. We expect our observations to offer a new perspective on learning better representations from data structures instead of human annotations for point cloud understanding.

Unsupervised Learning of 3D Structure from Images

Unsupervised 3D Shape Learning from Image Collections in the Wild

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

Unsupervised 3D Reconstruction from a Single Image via Adversarial Learning

Self-Supervised Learning for Ordered Three-Dimensional Structures

Unsupervised Learning of Visual 3D Keypoints for Control

Learning 3D Object Shape and Layout without 3D Supervision

Unsupervised Discovery of 3D Hierarchical Structure with Generative Diffusion Features

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches

Video Supervised for 3D Reconstruction from Single Image

Enhancing 2D Representation Learning with a 3D Prior

Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning 3D Scene Priors with 2D Supervision

Unsupervised Volumetric Animation

Learning to Infer Implicit Surfaces without 3D Supervision

Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images

PointGLR: Unsupervised Structural Representation Learning of 3D Point Clouds

Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching

Unsupervised Monocular Depth Reconstruction of Non-Rigid Scenes

Learning single-image 3D reconstruction by generative modelling of shape, pose and shading