Abstract:Diffusion models have been recognized for their ability to generate images that are not only visually appealing but also of high artistic quality. As a result, Layout-to-Image (L2I) generation has been proposed to leverage region-specific positions and descriptions to enable more precise and controllable generation. However, previous methods primarily focus on UNet-based models (e.g., SD1.5 and SDXL), and limited effort has explored Multimodal Diffusion Transformers (MM-DiTs), which have demonstrated powerful image generation capabilities. Enabling MM-DiT for layout-to-image generation seems straightforward but is challenging due to the complexity of how layout is introduced, integrated, and balanced among multiple modalities. To this end, we explore various network variants to efficiently incorporate layout guidance into MM-DiT, and ultimately present SiamLayout. To Inherit the advantages of MM-DiT, we use a separate set of network weights to process the layout, treating it as equally important as the image and text modalities. Meanwhile, to alleviate the competition among modalities, we decouple the image-layout interaction into a siamese branch alongside the image-text one and fuse them in the later stage. Moreover, we contribute a large-scale layout dataset, named LayoutSAM, which includes 2.7 million image-text pairs and 10.7 million entities. Each entity is annotated with a bounding box and a detailed description. We further construct the LayoutSAM-Eval benchmark as a comprehensive tool for evaluating the L2I generation quality. Finally, we introduce the Layout Designer, which taps into the potential of large language models in layout planning, transforming them into experts in layout generation and optimization. Our code, model, and dataset will be available at <a class="link-external link-https" href="https://creatilayout.github.io" rel="external noopener nofollow">this https URL</a>.

DensityLayout: Density-Conditioned Layout GAN for Visual-Textual Presentation Designs.

Composition-aware Graphic Layout GAN for Visual-textual Presentation Designs

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

Attribute-Conditioned Layout GAN for Automatic Graphic Design

PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layout

Element-conditioned GAN for graphic layout generation

LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators

Unsupervised Domain Adaption with Pixel-level Discriminator for Image-aware Layout Generation

LayoutGPT: Compositional Visual Planning and Generation with Large Language Models

Image-aware Layout Generation with User Constraints for Poster Design

Content-aware generative modeling of graphic design layouts

Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis

LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformer

LayoutDM: Transformer-based Diffusion Model for Layout Generation

Automatic Generation of Visual-Textual Presentation Layout.

CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation

Interactive Image Synthesis with Panoptic Layout Generation

Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis

Layout2image: Image Generation from Layout

Geometry Aligned Variational Transformer for Image-conditioned Layout Generation

LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation