Abstract:In the majority of GAN architectures, the latent space is defined as a set of vectors of given dimensionality. Such representations are not easily interpretable and do not capture spatial information of image content directly. In this work, we define a family of spatial latent spaces for StyleGAN2, capable of capturing more details and representing images that are out-of-sample in terms of the number and arrangement of object parts, such as an image of multiple faces or a face with more than two eyes. We propose a method for encoding images into our spaces, together with an attribute model capable of performing attribute editing in these spaces. We show that our spaces are effective for image manipulation and encode semantic information well. Our approach can be used on pre-trained generator models, and attribute edition can be done using pre-generated direction vectors making the barrier to entry for experimentation and use extremely low. We propose a regularization method for optimizing latent representations, which equalizes distributions of parts of latent spaces, making representations much closer to generated ones. We use it for encoding images into spatial spaces to obtain significant improvement in quality while keeping semantics and ability to use our attribute model for edition purposes. In total, using our methods gives encoding quality boost even as high as 30% in terms of LPIPS score comparing to standard methods, while keeping semantics. Additionally, we propose a StyleGAN2 training procedure on our spatial latent spaces, together with a custom spatial latent representation distribution to make spatially closer elements in the representation more dependent on each other than farther elements. Such approach improves the FID score by 29% on SpaceNet, and is able to generate consistent images of arbitrary sizes on spatially homogeneous datasets, like satellite imagery.

Improved StyleGAN Embedding: Where are the Good Latents?

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?

Collaborative Learning for Faster StyleGAN Embedding

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing

Style Transformer for Image Inversion and Editing

HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing

Designing an encoder for StyleGAN image manipulation

Semantic Unfolding of StyleGAN Latent Space

Towards Spatially Disentangled Manipulation of Face Images With Pre-Trained StyleGANs

Spatial Latent Representations in Generative Adversarial Networks for Image Generation

Make It So: Steering StyleGAN for Any Image Inversion and Editing

E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion

StyleRes: Transforming the Residuals for Real Image Editing with StyleGAN

High-Fidelity StyleGAN Inversion and Attribute Editing for Low Light Images

Trained Latent Space Navigation to Prevent Lack of Photorealism in Generated Images on Style-based Models

ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing

State‐of‐the‐Art in the Architecture, Methods and Applications of StyleGAN

Style Generator Inversion for Image Enhancement and Animation

HyperInverter: Improving StyleGAN Inversion via Hypernetwork