Abstract:The generator in the generative adversarial network (GAN) learns image generation in a coarse-to-fine manner in which earlier layers learn the overall structure of the image and the latter ones refine the details. To propagate the coarse information well, recent works usually build their generators by stacking up multiple residual blocks. Although the residual block can produce a high-quality image as well as be trained stably, it often impedes the information flow in the network. To alleviate this problem, this brief introduces a novel generator architecture that produces the image by combining features obtained through two different branches: the main and auxiliary branches. The goal of the main branch is to produce the image by passing through the multiple residual blocks, whereas the auxiliary branch is to convey the coarse information in the earlier layer to the later one. To combine the features in the main and auxiliary branches successfully, we also propose a gated feature fusion module (GFFM) that controls the information flow in those branches. To prove the superiority of the proposed method, this brief provides extensive experiments using various standard datasets including CIFAR-10, CIFAR-100, LSUN, CelebA-HQ, AFHQ, and tiny-ImageNet. Furthermore, we conducted various ablation studies to demonstrate the generalization ability of the proposed method. Quantitative evaluations prove that the proposed method exhibits impressive GAN performance in terms of Inception score (IS) and Frechet inception distance (FID). For instance, the proposed method boosts the FID and IS scores on the tiny-ImageNet dataset from 35.13 to 25.00 and 20.23 to 25.57, respectively.

MemoryGAN: GAN Generator As Heterogeneous Memory for Compositional Image Synthesis

Learning to Generate with Memory

MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

A Compact Memristor-based GAN Architecture with a Case Study on Single Image Super-Resolution

MFS: A Brain-Inspired Memory Formation System for GAN

DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis

Compositional GAN: Learning Conditional Image Composition

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

Memory Replay GANs: Learning to Generate Images from New Categories Without Forgetting

Generating Images Part by Part with Composite Generative Adversarial Networks

Compositional GAN: Learning Image-Conditional Binary Composition

Generative Modeling with Explicit Memory

Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation.

A Novel Generator With Auxiliary Branch for Improving GAN Performance

Few-shot Generation Via Recalling Brain-Inspired Episodic-Semantic Memory.

Hierarchy Composition GAN for High-fidelity Image Synthesis

Generating Memorable Images Based on Human Visual Memory Schemas

MF-GAN: Multi-conditional Fusion Generative Adversarial Network for Text-to-Image Synthesis

Adaptive Forgetting, Drafting and Comprehensive Guiding: Text-to-Image Synthesis with Hierarchical Generative Adversarial Networks

GAN Memory with No Forgetting