Abstract:Given a real source scene and a virtual target scene, the real-scene-constrained virtual scene layout synthesis problem is defined as how to re-synthesize the layout of the virtual furniture in the virtual scene to form a new virtual scene such that the new scene not only looks similar to the input real and virtual scenes but also is interactive. The goal of this problem is to maximize interactivity and fidelity which are contradictory. To solve this problem, we propose a real-scene-constrained virtual scene layout synthesis method to synthesize the layout of the virtual furniture in the new virtual scene. We split the scene layout synthesis process into 3 interrelated steps: scene matching, matched furniture layout generating, and unmatched furniture layout generating. For scene matching, we propose a deep scene matching network to predict the matching relationship between real and virtual furniture. For matched furniture layout generating, we propose a layout parameters optimization algorithm to predict suitable layouts of the matched virtual furniture. For unmatched furniture layout generating, we propose a deep scene generating network to predict suitable layouts of unmatched virtual furniture. We evaluate the quality of our method to synthesize scenes of different kinds and sizes. The results show that, compared with the heuristic rules-based method, our method has better matching accuracy and location accuracy. We also design a user study to evaluate the interactivity and fidelity. Compared to the manual method and the heuristic rules-based method, our method has a significant improvement in interactivity and fidelity.

Reshuffle-based Interior Scene Synthesis

DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-Aware Scene Synthesis

Structure Guided Interior Scene Synthesis Via Graph Matching.

Learning 3 D Scene Synthesis from Annotated RGB-D Images

Adaptive synthesis of indoor scenes via activity-associated object relation graphs.

SceneExpander: Real-Time Scene Synthesis for Interactive Floor Plan Editing

Action-driven 3D Indoor Scene Evolution

Automatic Generation of Commercial Scenes

Wall Grid Structure for Interior Scene Synthesis.

FuncScene: Function-centric Indoor Scene Synthesis via a Variational AutoEncoder Framework

Real-scene-constrained virtual scene layout synthesis for mixed reality

CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding

Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extraction

SceneExplorer: an Interactive System for Expanding, Scheduling, and Organizing Transformable Layouts

Style-compatible Object Recommendation for Multi-room Indoor Scene Synthesis

Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects

SceneFormer: Indoor Scene Generation with Transformers

GLTScene: Global‐to‐Local Transformers for Indoor Scene Synthesis with General Room Boundaries

ATISS: Autoregressive Transformers for Indoor Scene Synthesis

RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation

Rearrange Indoor Scenes for Human-Robot Co-Activity