Abstract:Significant progress has been made in training large generative models for natural language and images. Yet, the advancement of 3D generative models is hindered by their substantial resource demands for training, along with inefficient, non-compact, and less expressive representations. This paper introduces Make-A-Shape, a new 3D generative model designed for efficient training on a vast scale, capable of utilizing 10 millions publicly-available shapes. Technical-wise, we first innovate a wavelet-tree representation to compactly encode shapes by formulating the subband coefficient filtering scheme to efficiently exploit coefficient relations. We then make the representation generatable by a diffusion model by devising the subband coefficients packing scheme to layout the representation in a low-resolution grid. Further, we derive the subband adaptive training strategy to train our model to effectively learn to generate coarse and detail wavelet coefficients. Last, we extend our framework to be controlled by additional input conditions to enable it to generate shapes from assorted modalities, e.g., single/multi-view images, point clouds, and low-resolution voxels. In our extensive set of experiments, we demonstrate various applications, such as unconditional generation, shape completion, and conditional generation on a wide range of modalities. Our approach not only surpasses the state of the art in delivering high-quality results but also efficiently generates shapes within a few seconds, often achieving this in just 2 seconds for most conditions. Our source code is available at <a class="link-external link-https" href="https://github.com/AutodeskAILab/Make-a-Shape" rel="external noopener nofollow">this https URL</a>.

ShapeCreator: 3D Shape Generation from Isomorphic Datasets Based on Autoencoder.

Deep Learning Representation using Autoencoder for 3D Shape Retrieval

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval

FullFormer: Generating Shapes Inside Shapes

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation

IsoExplorer: an Isosurface-Driven Framework for 3D Shape Analysis of Biomedical Volume Data

Progressive Shape-Distribution-Encoder for Learning 3D Shape Representation

Unsupervised Representation Learning for Diverse Deformable Shape Collections

Learning to Generate 3D Shapes from a Single Example

3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability

Progressive Shape-Distribution-Encoder for 3D Shape Retrieval

DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation

Make-A-Shape: a Ten-Million-scale 3D Shape Model

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Parameterize Structure with Differentiable Template for 3D Shape Generation

Compress3D: a Compressed Latent Space for 3D Generation from a Single Image

Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation

Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes

3D Shape Augmentation with Content-Aware Shape Resizing

PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers

Multi-chart Generative Surface Modeling