Abstract:Given a real source scene and a virtual target scene, the real-scene-constrained virtual scene layout synthesis problem is defined as how to re-synthesize the layout of the virtual furniture in the virtual scene to form a new virtual scene such that the new scene not only looks similar to the input real and virtual scenes but also is interactive. The goal of this problem is to maximize interactivity and fidelity which are contradictory. To solve this problem, we propose a real-scene-constrained virtual scene layout synthesis method to synthesize the layout of the virtual furniture in the new virtual scene. We split the scene layout synthesis process into 3 interrelated steps: scene matching, matched furniture layout generating, and unmatched furniture layout generating. For scene matching, we propose a deep scene matching network to predict the matching relationship between real and virtual furniture. For matched furniture layout generating, we propose a layout parameters optimization algorithm to predict suitable layouts of the matched virtual furniture. For unmatched furniture layout generating, we propose a deep scene generating network to predict suitable layouts of unmatched virtual furniture. We evaluate the quality of our method to synthesize scenes of different kinds and sizes. The results show that, compared with the heuristic rules-based method, our method has better matching accuracy and location accuracy. We also design a user study to evaluate the interactivity and fidelity. Compared to the manual method and the heuristic rules-based method, our method has a significant improvement in interactivity and fidelity.

GLTScene: Global‐to‐Local Transformers for Indoor Scene Synthesis with General Room Boundaries

Learning 3 D Scene Synthesis from Annotated RGB-D Images

RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation

SceneFormer: Indoor Scene Generation with Transformers

End-to-end Generative Floor-plan and Layout with Attributes and Relation Graph

Adaptive synthesis of indoor scenes via activity-associated object relation graphs.

FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenes

LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Data

Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction

Wall Grid Structure for Interior Scene Synthesis.

Non-autoregressive transformer with fine-grained optimization for user-specified indoor layout

CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding

Human-centric Indoor Scene Synthesis Using Stochastic Grammar

LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model

ATISS: Autoregressive Transformers for Indoor Scene Synthesis

Flexible Indoor Scene Synthesis Via a Multi-object Particle Swarm Intelligence Optimization Algorithm and User Intentions.

Structure Guided Interior Scene Synthesis Via Graph Matching.

Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

Real-scene-constrained virtual scene layout synthesis for mixed reality

3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics