Abstract:We present a significant breakthrough in 3D shape generation by scaling it to unprecedented dimensions. Through the adaptation of the Auto-Regressive model and the utilization of large language models, we have developed a remarkable model with an astounding 3.6 billion trainable parameters, establishing it as the largest 3D shape generation model to date, named Argus-3D. Our approach addresses the limitations of existing methods by enhancing the quality and diversity of generated 3D shapes. To tackle the challenges of high-resolution 3D shape generation, our model incorporates tri-plane features as latent representations, effectively reducing computational complexity. Additionally, we introduce a discrete codebook for efficient quantization of these representations. Leveraging the power of transformers, we enable multi-modal conditional generation, facilitating the production of diverse and visually impressive 3D shapes. To train our expansive model, we leverage an ensemble of publicly-available 3D datasets, consisting of a comprehensive collection of approximately 900,000 objects from renowned repositories such as ModelNet40, ShapeNet, Pix3D, 3D-Future, and Objaverse. This diverse dataset empowers our model to learn from a wide range of object variations, bolstering its ability to generate high-quality and diverse 3D shapes. Extensive experimentation demonstrate the remarkable efficacy of our approach in significantly improving the visual quality of generated 3D shapes. By pushing the boundaries of 3D generation, introducing novel methods for latent representation learning, and harnessing the power of transformers for multi-modal conditional generation, our contributions pave the way for substantial advancements in the field. Our work unlocks new possibilities for applications in gaming, virtual reality, product design, and other domains that demand high-quality and diverse 3D objects.

3D Shape Augmentation with Content-Aware Shape Resizing

Quantitative and Flexible 3D Shape Dataset Augmentation Via Latent Space Embedding and Deformation Learning

3D-VirtFusion: Synthetic 3D Data Augmentation through Generative Diffusion Models and Controllable Editing

Pushing the Limits of 3D Shape Generation at Scale

“Edutainment 2017” a Visual and Semantic Representation of 3D Face Model for Reshaping Face in Images

Make-A-Shape: a Ten-Million-scale 3D Shape Model

NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation

An Augmentation Strategy for Medical Image Processing Based on Statistical Shape Model and 3D Thin Plate Spline for Deep Learning

FullFormer: Generating Shapes Inside Shapes

Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes

Learning to Infer Semantic Parameters for 3D Shape Editing

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval

3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability

Twin-Channel Gan: Repair Shape with Twin-Channel Generative Adversarial Network and Structural Constraints

SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation

Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability

Leveraging Smooth Deformation Augmentation for LiDAR Point Cloud Semantic Segmentation

DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement

A Semi-Supervised Data Augmentation Approach using 3D Graphical Engines

FAME: 3D Shape Generation via Functionality-Aware Model Evolution

Bridging Formal Shape Models and Deep Learning: A Novel Fusion for Understanding 3D Objects