Abstract:The latent space of a Generative Adversarial Network (GAN) has been shown to encode rich semantics within some subspaces. To identify these subspaces, researchers typically analyze the statistical information from a collection of synthesized data, and the identified subspaces tend to control image attributes globally (i.e., manipulating an attribute causes the change of an entire image). By contrast, this work introduces low-rank subspaces that enable more precise control of GAN generation. Concretely, given an arbitrary image and a region of interest (e.g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces. There are three distinguishable strengths of our approach that can be aptly called LowRankGAN. First, compared to analytic algorithms in prior work, our low-rank factorization of Jacobians is able to find the low-dimensional representation of attribute manifold, making image editing more precise and controllable. Second, low-rank factorization naturally yields a null space of attributes such that moving the latent code within it only affects the outer region of interest. Therefore, local image editing can be simply achieved by projecting an attribute vector into the null space without relying on a spatial mask as existing methods do. Third, our method can robustly work with a local region from one image for analysis yet well generalize to other images, making it much easy to use in practice. Extensive experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.

Understanding Noise Injection in GANs Supplementary Material

Understanding Noise Injection in GANs.

On Noise Injection in Generative Adversarial Networks

Noise Dimension of GAN: An Image Compression Perspective

Low-Rank Subspaces in GANs

Hausdorff GAN: Improving GAN Generation Quality With Hausdorff Metric

Uncertainty Principles of Encoding GANs Supplementary Material

Understanding the Effectiveness of Lipschitz Constraint in Training of GANs Via Gradient Analysis.

Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack

Understanding GANs: the LQG Setting

Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation

Towards a Better Understanding and Regularization of GAN Training Dynamics

On Computation and Generalization of GANs with Spectrum Control

GAN: Dynamics

Towards a Better Global Loss Landscape of GANs

Quantization of Generative Adversarial Networks for Efficient Inference: a Methodological Study

Revealing the Utilized Rank of Subspaces of Learning in Neural Networks

Towards the Gradient Vanishing, Divergence Mismatching and Mode Collapse of Generative Adversarial Nets

Inception Score, Label Smoothing, Gradient Vanishing and -log(D(x)) Alternative

Soft-margin Ellipsoid generative adversarial networks

Supplementary Materials: Patch-wise Attack for Fooling Deep Neural Network