Abstract:We present a novel framework for multi-domain synthesis of artwork from semantic layouts. One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis. To address this problem, we propose a dataset, which we call ArtSem, that contains 40,000 images of artwork from 4 different domains with their corresponding semantic label maps. We generate the dataset by first extracting semantic maps from landscape photography and then propose a conditional Generative Adversarial Network (GAN)-based approach to generate high-quality artwork from the semantic maps without necessitating paired training data. Furthermore, we propose an artwork synthesis model that uses domain-dependent variational encoders for high-quality multi-domain synthesis. The model is improved and complemented with a simple but effective normalization method, based on normalizing both the semantic and style jointly, which we call Spatially STyle-Adaptive Normalization (SSTAN). In contrast to previous methods that only take semantic layout as input, our model is able to learn a joint representation of both style and semantic information, which leads to better generation quality for synthesizing artistic images. Results indicate that our model learns to separate the domains in the latent space, and thus, by identifying the hyperplanes that separate the different domains, we can also perform fine-grained control of the synthesized artwork. By combining our proposed dataset and approach, we are able to generate user-controllable artwork that is of higher quality than existing

Semantic Image Synthesis Via Hierarchical Structure Features

Fine-grained Semantic Constraint in Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Semantic Image Synthesis with Unconditional Generator

Unlocking Pre-trained Image Backbones for Semantic Image Synthesis

Semantic Probability Distribution Modeling for Diverse Semantic Image Synthesis

Synthetic Convolutional Features for Improved Semantic Segmentation

SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generation

Diverse Semantic Image Synthesis via Probability Distribution Modeling

Semantic Guided Level-Category Hybrid Prediction Network for Hierarchical Image Classification.

Perceptual Pyramid Adversarial Networks for Text-to-Image Synthesis.

Semantic Image Synthesis Via Diffusion Models

Semantic Image Synthesis via Class-Adaptive Cross-Attention

Semantic RGB-D Image Synthesis

IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis

Semantic-aware Network for Aerial-to-Ground Image Synthesis

Semantic Image Synthesis via Adversarial Learning

Semantic Image Synthesis with Spatially-Adaptive Normalization

Controllable Multi-domain Semantic Artwork Synthesis

Semi-parametric Image Synthesis

FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models