Abstract:In the majority of GAN architectures, the latent space is defined as a set of vectors of given dimensionality. Such representations are not easily interpretable and do not capture spatial information of image content directly. In this work, we define a family of spatial latent spaces for StyleGAN2, capable of capturing more details and representing images that are out-of-sample in terms of the number and arrangement of object parts, such as an image of multiple faces or a face with more than two eyes. We propose a method for encoding images into our spaces, together with an attribute model capable of performing attribute editing in these spaces. We show that our spaces are effective for image manipulation and encode semantic information well. Our approach can be used on pre-trained generator models, and attribute edition can be done using pre-generated direction vectors making the barrier to entry for experimentation and use extremely low. We propose a regularization method for optimizing latent representations, which equalizes distributions of parts of latent spaces, making representations much closer to generated ones. We use it for encoding images into spatial spaces to obtain significant improvement in quality while keeping semantics and ability to use our attribute model for edition purposes. In total, using our methods gives encoding quality boost even as high as 30% in terms of LPIPS score comparing to standard methods, while keeping semantics. Additionally, we propose a StyleGAN2 training procedure on our spatial latent spaces, together with a custom spatial latent representation distribution to make spatially closer elements in the representation more dependent on each other than farther elements. Such approach improves the FID score by 29% on SpaceNet, and is able to generate consistent images of arbitrary sizes on spatially homogeneous datasets, like satellite imagery.

Surrogate Gradient Field for Latent Space Manipulation

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search

L2M-GAN: Learning to Manipulate Latent Space Semantics for Facial Attribute Editing

Spatial Attention Guided Local Facial Attribute Editing

Spatially Constrained Generative Adversarial Networks for Conditional Image Generation

Towards Spatially Disentangled Manipulation of Face Images With Pre-Trained StyleGANs

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation

Interpreting the Latent Space of GANs for Semantic Face Editing

Controllable facial attribute editing via Gaussian mixture model disentanglement

Self-Conditioned GANs for Image Editing

Self-Conditioned Generative Adversarial Networks for Image Editing

LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract

FINE-GRAINED EXPRESSION MANIPULATION VIA STRUCTURED LATENT SPACE

Enhancing Conditional Image Generation with Explainable Latent Space Manipulation

SC-GAN: Subspace Clustering Based GAN for Automatic Expression Manipulation.

FEAT: Face Editing with Attention

Unsupervised Image Transformation Learning via Generative Adversarial Networks

Adaptive Nonlinear Latent Transformation for Conditional Face Editing

Disentangling the Latent Space of GANs for Semantic Face Editing.

Spatial Latent Representations in Generative Adversarial Networks for Image Generation

Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation