Abstract:In the majority of GAN architectures, the latent space is defined as a set of vectors of given dimensionality. Such representations are not easily interpretable and do not capture spatial information of image content directly. In this work, we define a family of spatial latent spaces for StyleGAN2, capable of capturing more details and representing images that are out-of-sample in terms of the number and arrangement of object parts, such as an image of multiple faces or a face with more than two eyes. We propose a method for encoding images into our spaces, together with an attribute model capable of performing attribute editing in these spaces. We show that our spaces are effective for image manipulation and encode semantic information well. Our approach can be used on pre-trained generator models, and attribute edition can be done using pre-generated direction vectors making the barrier to entry for experimentation and use extremely low. We propose a regularization method for optimizing latent representations, which equalizes distributions of parts of latent spaces, making representations much closer to generated ones. We use it for encoding images into spatial spaces to obtain significant improvement in quality while keeping semantics and ability to use our attribute model for edition purposes. In total, using our methods gives encoding quality boost even as high as 30% in terms of LPIPS score comparing to standard methods, while keeping semantics. Additionally, we propose a StyleGAN2 training procedure on our spatial latent spaces, together with a custom spatial latent representation distribution to make spatially closer elements in the representation more dependent on each other than farther elements. Such approach improves the FID score by 29% on SpaceNet, and is able to generate consistent images of arbitrary sizes on spatially homogeneous datasets, like satellite imagery.

Stylistic scene enhancement GAN: mixed stylistic enhancement generation for 3D indoor scenes

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Statistics Enhancement Generative Adversarial Networks for Diverse Conditional Image Synthesis

Creative and Diverse Artwork Generation Using Adversarial Networks

Pscenegan: Multi-Domain Particular Scenes Generation Based on Conditional Generative Adversarial Networks

Style recommendation and simulation for handmade artworks using generative adversarial networks

Style Intervention: How to Achieve Spatial Disentanglement with Style-based Generators?

Indoor Scene Generation from a Collection of Semantic-Segmented Depth Images

Towards Spatially Disentangled Manipulation of Face Images With Pre-Trained StyleGANs

Spatial Fusion GAN for Image Synthesis

Style Separation and Synthesis Via Generative Adversarial Networks

BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation

Semantic 3D Indoor Scene Enhancement Using Guide Words

Generative Image Modeling using Style and Structure Adversarial Networks

SDP-GAN: Saliency Detail Preservation Generative Adversarial Networks for High Perceptual Quality Style Transfer

Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation

Spatial Latent Representations in Generative Adversarial Networks for Image Generation

Structured Generative Adversarial Networks.

Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption

Exploiting Relationship for Complex-scene Image Generation

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer