GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation

Ying Liu,Wenhong Cai,Xiaohui Yuan,Jinhai Xiang
DOI: https://doi.org/10.48550/arXiv.2008.02436
2020-08-06
Abstract:Although Generative Adversarial Networks have shown remarkable performance in image generation, there are some challenges in image realism and convergence speed. The results of some models display the imbalances of quality within a generated image, in which some defective parts appear compared with other regions. Different from general single global optimization methods, we introduce an adaptive global and local bilevel optimization model(GL-GAN). The model achieves the generation of high-resolution images in a complementary and promoting way, where global optimization is to optimize the whole images and local is only to optimize the low-quality areas. With a simple network structure, GL-GAN is allowed to effectively avoid the nature of imbalance by local bilevel optimization, which is accomplished by first locating low-quality areas and then optimizing them. Moreover, by using feature map cues from discriminator output, we propose the adaptive local and global optimization method(Ada-OP) for specific implementation and find that it boosts the convergence speed. Compared with the current GAN methods, our model has shown impressive performance on CelebA, CelebA-HQ and LSUN datasets.
Computer Vision and Pattern Recognition,Image and Video Processing
What problem does this paper attempt to address?
This paper attempts to solve two main problems existing in Generative Adversarial Networks (GANs) in image generation: image authenticity and convergence speed. Specifically: 1. **Image Authenticity**: Although GANs perform well in image generation, in the generated images, there may be an imbalance in the quality of certain regions. For example, when some models synthesize images, the overall structure (such as facial contours, eye positions, and hairstyles, etc.) is well - presented, but there are problems in details (such as blemishes, distortions, or不协调 regions). This may be because common global optimization models ignore the optimization of small - scale low - quality regions, resulting in these regions not being fully improved. 2. **Convergence Speed**: Existing GAN models may face the problem of slow convergence speed during the training process, especially when dealing with high - resolution images. To solve these problems, the paper proposes an Adaptive Global and Local Two - layer Optimization Model (GL - GAN). By combining global optimization and local optimization, this model can generate high - quality images more effectively and improve the convergence speed. Specifically: - **Global Optimization**: Optimize the entire image to ensure the overall quality of the image. - **Local Optimization**: Specifically optimize low - quality regions, avoiding repeated optimization of the entire image, thus improving efficiency. In addition, the author also introduces an Adaptive Local and Global Optimization Method (Ada - OP), which locates and optimizes low - quality regions through clues in the feature map, further enhancing the performance of the model. ### Mathematical Formula Summary 1. **Discriminator Output**: \[ y_{h\times w}=D_{\phi}(x) \] where \(D_{\phi}\) represents the discriminator with parameter \(\phi\), \(x\) is the sample, and \(y_{h\times w}\in\mathbb{R}^{h\times w}\) is a matrix, and each element corresponds to the quality assessment of a receptive field of the image. 2. **Global Optimization Objective**: \[ \text{Object 2}=\min_{\theta}\mathbb{E}_{z\sim P_{z}}[f(-D_{\phi}(G_{\theta}(z)))] \] 3. **Local Two - layer Optimization Model**: \[ \text{Object 1}=\min_{\theta}\mathbb{E}_{z\sim P_{z}}[f(h^{*}\odot(-D_{\phi}(G_{\theta}(z))))] \] where \(h^{*}\) is the optimal mask matrix, selected through inner - layer optimization: \[ h^{*}\in\arg\max_{h}\sum_{i,j}(h\odot(D_{\phi}(G_{\theta}(z))-\alpha)) \] 4. **Adaptive Global and Local Optimization Method (Ada - OP)**: \[ \text{Objective}= \begin{cases} \text{Object 2}&\text{if }\sigma\geq\beta\\ \text{Object 1, where }\alpha = \alpha_{1}&\text{if }\bar{\sigma}\leq\delta_{1}\\ \text{Object 1, where }\alpha = \alpha_{2}&\text{if }\delta_{1}\leq\bar{\sigma}\leq\delta_{2}\\ \text{Object 1, where }\alpha = \alpha_{3}&\text{if }\bar{\sigma}\geq\delta_{2} \end{cases} \] Through these methods, GL - GAN in C