Abstract:Generating multi-view images based on text or single-image prompts is a critical capability for the creation of 3D content. Two fundamental questions on this topic are what data we use for training and how to ensure multi-view consistency. This paper introduces a novel framework that makes fundamental contributions to both questions. Unlike leveraging images from 2D diffusion models for training, we propose a dense consistent multi-view generation model that is fine-tuned from off-the-shelf video generative models. Images from video generative models are more suitable for multi-view generation because the underlying network architecture that generates them employs a temporal module to enforce frame consistency. Moreover, the video data sets used to train these models are abundant and diverse, leading to a reduced train-finetuning domain gap. To enhance multi-view consistency, we introduce a 3D-Aware Denoising Sampling, which first employs a feed-forward reconstruction module to get an explicit global 3D model, and then adopts a sampling strategy that effectively involves images rendered from the global 3D model into the denoising sampling loop to improve the multi-view consistency of the final images. As a by-product, this module also provides a fast way to create 3D assets represented by 3D Gaussians within a few seconds. Our approach can generate 24 dense views and converges much faster in training than state-of-the-art approaches (4 GPU hours versus many thousand GPU hours) with comparable visual quality and consistency. By further fine-tuning, our approach outperforms existing state-of-the-art methods in both quantitative metrics and visual effects. Our project page is

Multi-View Consistent Generative Adversarial Networks for 3D-Aware Image Synthesis

Multi-view Consistent Generative Adversarial Networks for Compositional 3D-Aware Image Synthesis

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

SDF-based 3D Generative Adversarial Networks for Image and Geometry Generation

Learning Multi-View Manifold for Single Image Based Modeling

Multi3D: 3D-Aware Multimodal Image Synthesis

Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models

View Independent Generative Adversarial Network for Novel View Synthesis

VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model

Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis

MV-Adapter: Multi-view Consistent Image Generation Made Easy

Towards Realistic 3D Embedding via View Alignment

G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis

SideGAN: 3D-Aware Generative Model for Improved Side-View Image Synthesis

Consistent123: Improve Consistency for One Image to 3D Object Synthesis

GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators

3D-aware Image Generation and Editing with Multi-modal Conditions

2D GANs Meet Unsupervised Single-view 3D Reconstruction

Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitation