Abstract:Significant progress has been made in training large generative models for natural language and images. Yet, the advancement of 3D generative models is hindered by their substantial resource demands for training, along with inefficient, non-compact, and less expressive representations. This paper introduces Make-A-Shape, a new 3D generative model designed for efficient training on a vast scale, capable of utilizing 10 millions publicly-available shapes. Technical-wise, we first innovate a wavelet-tree representation to compactly encode shapes by formulating the subband coefficient filtering scheme to efficiently exploit coefficient relations. We then make the representation generatable by a diffusion model by devising the subband coefficients packing scheme to layout the representation in a low-resolution grid. Further, we derive the subband adaptive training strategy to train our model to effectively learn to generate coarse and detail wavelet coefficients. Last, we extend our framework to be controlled by additional input conditions to enable it to generate shapes from assorted modalities, e.g., single/multi-view images, point clouds, and low-resolution voxels. In our extensive set of experiments, we demonstrate various applications, such as unconditional generation, shape completion, and conditional generation on a wide range of modalities. Our approach not only surpasses the state of the art in delivering high-quality results but also efficiently generates shapes within a few seconds, often achieving this in just 2 seconds for most conditions. Our source code is available at <a class="link-external link-https" href="https://github.com/AutodeskAILab/Make-a-Shape" rel="external noopener nofollow">this https URL</a>.

Wavelet transform-assisted generative model for efficient 3d deep shape generation

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

3D-WAG: Hierarchical Wavelet-Guided Autoregressive Generation for High-Fidelity 3D Shapes

Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis

Learning Progressive Point Embeddings for 3D Point Cloud Generation

PointWavelet: Learning in Spectral Domain for 3D Point Cloud Analysis

PointWavelet: Learning in Spectral Domain for 3-D Point Cloud Analysis

Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation

Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

Make-A-Shape: a Ten-Million-scale 3D Shape Model

Neural Volumetric Mesh Generator

A Geometry Aware Diffusion Model for 3D Point Cloud Generation

Learning Energy-Based 3D Descriptor Networks for Volumetric Shape Synthesis and Analysis

Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds

Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

Learning to Generate 3D Shapes from a Single Example

FullFormer: Generating Shapes Inside Shapes

Deep Optimized Priors for 3D Shape Modeling and Reconstruction

Progressive Generation of 3D Point Clouds with Hierarchical Consistency

A survey of deep learning-based 3D shape generation