Abstract:The significant progress on Generative Adversarial Networks (GANs) has facilitated realistic single-object image generation based on language input. However, complex-scene generation (with various interactions among multiple objects) still suffers from messy layouts and object distortions, due to diverse configurations in layouts and appearances. Prior methods are mostly object-driven and ignore their inter-relations that play a significant role in complex-scene images. This work explores relationship-aware complex-scene image generation, where multiple objects are inter-related as a scene graph. With the help of relationships, we propose three major updates in the generation framework. First, reasonable spatial layouts are inferred by jointly considering the semantics and relationships among objects. Compared to standard location regression, we show relative scales and distances serve a more reliable target. Second, since the relations between objects significantly influence an object's appearance, we design a relation-guided generator to generate objects reflecting their relationships. Third, a novel scene graph discriminator is proposed to guarantee the consistency between the generated image and the input scene graph. Our method tends to synthesize plausible layouts and objects, respecting the interplay of multiple objects in an image. Experimental results on Visual Genome and HICO-DET datasets show that our proposed method significantly outperforms prior arts in terms of IS and FID metrics. Based on our user study and visual inspection, our method is more effective in generating logical layout and appearance for complex-scenes.

Using Scene Graph Context to Improve Image Generation

Learning Canonical Representations for Scene Graph to Image Generation

Attention Redirection Transformer with Semantic Oriented Learning for Unbiased Scene Graph Generation

Scene Graph Generation with Geometric Context

Image Generation from Scene Graph with Object Edges

Unconditional Scene Graph Generation

Rethinking Image Generation from Scene Graphs with Attention Mechanism

Counterfactual Critic Multi-Agent Training for Scene Graph Generation

Generating Triples with Adversarial Networks for Scene Graph Construction

Scene Graph Generation With Hierarchical Context

Exploiting Relationship for Complex-scene Image Generation

Scene Graph Generation Based On Node-Relation Context Module

Scene Graph Generation for Better Image Captioning?

Scene Dynamics: Counterfactual Critic Multi-Agent Training for Scene Graph Generation.

Fast Contextual Scene Graph Generation with Unbiased Context Augmentation.

What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation

Scene Graph to Image Synthesis Via Knowledge Consensus.

Graph R-CNN for Scene Graph Generation

Deep Generative Probabilistic Graph Neural Networks for Scene Graph Generation

Joint Generative Modeling of Scene Graphs and Images via Diffusion Models

Image Synthesis with Graph Conditioning: CLIP-Guided Diffusion Models for Scene Graphs