Abstract:Abstract We present a novel framework for the multi-domain synthesis of artworks from semantic layouts. One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis. To address this problem, we propose a dataset called ArtSem that contains 40,000 images of artwork from four different domains, with their corresponding semantic label maps. We first extracted semantic maps from landscape photography and used a conditional generative adversarial network (GAN)-based approach for generating high-quality artwork from semantic maps without requiring paired training data. Furthermore, we propose an artwork-synthesis model using domain-dependent variational encoders for high-quality multi-domain synthesis. Subsequently, the model was improved and complemented with a simple but effective normalization method based on jointly normalizing semantics and style, which we call spatially style-adaptive normalization (SSTAN). Compared to the previous methods, which only take semantic layout as the input, our model jointly learns style and semantic information representation, improving the generation quality of artistic images. These results indicate that our model learned to separate the domains in the latent space. Thus, we can perform fine-grained control of the synthesized artwork by identifying hyperplanes that separate the different domains. Moreover, by combining the proposed dataset and approach, we generated user-controllable artworks of higher quality than that of existing approaches, as corroborated by quantitative metrics and a user study.

Semantic Probability Distribution Modeling for Diverse Semantic Image Synthesis

Diverse Semantic Image Synthesis via Probability Distribution Modeling

Fine-grained Semantic Constraint in Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Semantic Image Synthesis Via Diffusion Models

On the Diversity of Conditional Image Synthesis with Semantic Layouts.

Semantic RGB-D Image Synthesis

Semantic-aware Noise Driven Portrait Synthesis and Manipulation

Semantically Multi-Modal Image Synthesis

On the Diversity of Realistic Image Synthesis.

IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis

Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis

Semantics Disentangling for Text-to-Image Generation

Dual Attention GANs for Semantic Image Synthesis

Semantic Image Synthesis with Unconditional Generator

Unlocking Pre-trained Image Backbones for Semantic Image Synthesis

Reconciling Semantic Controllability and Diversity for Remote Sensing Image Synthesis with Hybrid Semantic Embedding

Semantic Image Synthesis via Adversarial Learning

Semantic Palette: Guiding Scene Generation with Class Proportions

Controllable multi-domain semantic artwork synthesis