Abstract:3D scene generation is in high demand across various domains, including virtual reality, gaming, and the film industry. Owing to the powerful generative capabilities of text-to-image diffusion models that provide reliable priors, the creation of 3D scenes using only text prompts has become viable, thereby significantly advancing researches in text-driven 3D scene generation. In order to obtain multiple-view supervision from 2D diffusion models, prevailing methods typically employ the diffusion model to generate an initial local image, followed by iteratively outpainting the local image using diffusion models to gradually generate scenes. Nevertheless, these outpainting-based approaches prone to produce global inconsistent scene generation results without high degree of completeness, restricting their broader applications. To tackle these problems, we introduce HoloDreamer, a framework that first generates high-definition panorama as a holistic initialization of the full 3D scene, then leverage 3D Gaussian Splatting (3D-GS) to quickly reconstruct the 3D scene, thereby facilitating the creation of view-consistent and fully enclosed 3D scenes. Specifically, we propose Stylized Equirectangular Panorama Generation, a pipeline that combines multiple diffusion models to enable stylized and detailed equirectangular panorama generation from complex text prompts. Subsequently, Enhanced Two-Stage Panorama Reconstruction is introduced, conducting a two-stage optimization of 3D-GS to inpaint the missing region and enhance the integrity of the scene. Comprehensive experiments demonstrated that our method outperforms prior works in terms of overall visual consistency and harmony as well as reconstruction quality and rendering robustness when generating fully enclosed scenes.

RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion

DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting

DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modeling

HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions

DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling

SceneDreamer360: Text-Driven 3D-Consistent Scene Generation with Panoramic Gaussian Splatting

3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation

DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion

Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting

EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior

SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections

VividDream: Generating 3D Scene with Ambient Dynamics

PaintScene4D: Consistent 4D Scene Generation from Text Prompts

Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Model

GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with Point Cloud Priors

PlacidDreamer: Advancing Harmony in Text-to-3D Generation

LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes

3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models

RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture