Abstract:Given a real source scene and a virtual target scene, the real-scene-constrained virtual scene layout synthesis problem is defined as how to re-synthesize the layout of the virtual furniture in the virtual scene to form a new virtual scene such that the new scene not only looks similar to the input real and virtual scenes but also is interactive. The goal of this problem is to maximize interactivity and fidelity which are contradictory. To solve this problem, we propose a real-scene-constrained virtual scene layout synthesis method to synthesize the layout of the virtual furniture in the new virtual scene. We split the scene layout synthesis process into 3 interrelated steps: scene matching, matched furniture layout generating, and unmatched furniture layout generating. For scene matching, we propose a deep scene matching network to predict the matching relationship between real and virtual furniture. For matched furniture layout generating, we propose a layout parameters optimization algorithm to predict suitable layouts of the matched virtual furniture. For unmatched furniture layout generating, we propose a deep scene generating network to predict suitable layouts of unmatched virtual furniture. We evaluate the quality of our method to synthesize scenes of different kinds and sizes. The results show that, compared with the heuristic rules-based method, our method has better matching accuracy and location accuracy. We also design a user study to evaluate the interactivity and fidelity. Compared to the manual method and the heuristic rules-based method, our method has a significant improvement in interactivity and fidelity.

SceneExpander: Real-Time Scene Synthesis for Interactive Floor Plan Editing

DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-Aware Scene Synthesis

Action-driven 3D Indoor Scene Evolution

SceneDirector: Interactive Scene Synthesis by Simultaneously Editing Multiple Objects in Real-Time

SceneExplorer: an Interactive System for Expanding, Scheduling, and Organizing Transformable Layouts

Learning 3 D Scene Synthesis from Annotated RGB-D Images

Real-scene-constrained virtual scene layout synthesis for mixed reality

Automatic Generation of Commercial Scenes

PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI

RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture

Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction

MageAdd: Real-Time Interaction Simulation for Scene Synthesis

Scene-Planning System Based on Augmented Reality Technique

Reshuffle-based Interior Scene Synthesis

SceneCraft: Layout-Guided 3D Scene Generation

Planner3D: LLM-enhanced graph prior meets 3D indoor scene explicit regularization

Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects

FuncScene: Function-centric Indoor Scene Synthesis via a Variational AutoEncoder Framework

Wall Grid Structure for Interior Scene Synthesis.

3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting

ATISS: Autoregressive Transformers for Indoor Scene Synthesis