Abstract:Image inpainting that completes large free-form missing regions in images is a promising yet challenging task. State-of-the-art approaches have achieved significant progress by taking advantage of generative adversarial networks (GAN). However, these approaches can suffer from generating distorted structures and blurry textures in high-resolution images (e.g., 512×512). The challenges mainly drive from (1) image content reasoning from distant contexts, and (2) fine-grained texture synthesis for a large missing region. To overcome these two challenges, we propose an enhanced GAN-based model, named Aggregated COntextual-Transformation GAN (AOT-GAN), for high-resolution image inpainting. Specifically, to enhance context reasoning, we construct the generator of AOT-GAN by stacking multiple layers of a proposed AOT block. The AOT blocks aggregate contextual transformations from various receptive fields, allowing to capture both informative distant image contexts and rich patterns of interest for context reasoning. For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task. Such a training objective forces the discriminator to distinguish the detailed appearances of real and synthesized patches, and in turn facilitates the generator to synthesize clear textures. Extensive comparisons on Places2, the most challenging benchmark with 1.8 million high-resolution images of 365 complex scenes, show that our model outperforms the state-of-the-art. A user study including more than 30 subjects further validates the superiority of AOT-GAN. We further evaluate the proposed AOT-GAN in practical applications, e.g., logo removal, face editing, and object removal. Results show that our model achieves promising completions in the real world. We release codes and models in https://github.com/researchmm/AOT-GAN-for-Inpainting.

GlueGAN: Gluing Two Images As a Panorama with Adversarial Learning

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Cross-View Panorama Image Synthesis.

SmartPaint: a Co-Creative Drawing System Based on Generative Adversarial Networks

Continuation of Famous Art with AI: A Conditional Adversarial Network Inpainting Approach

A three-stage GAN model based on edge and color prediction for image outpainting

DLP-GAN: learning to draw modern Chinese landscape photos with generative adversarial network

2D-to-Stereo Panorama Conversion Using GAN and Concentric Mosaics.

DGCA: high resolution image inpainting via DR-GAN and contextual attention

HI-GAN: Hierarchical Inpainting GAN with Auxiliary Inputs for Combined RGB and Depth Inpainting

Viewport-Oriented Panoramic Image Inpainting

Aggregated Contextual Transformations for High-Resolution Image Inpainting

DE-GAN: Domain Embedded GAN for High Quality Face Image Inpainting

PIINET: A 360-degree Panoramic Image Inpainting Network Using a Cube Map

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing

Eigan: Enhanced Inpainting Generative Adversarial Network.

Panoramic Image Inpainting With Gated Convolution And Contextual Reconstruction Loss

End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks

A Generative Adversarial Framework for Optimizing Image Matting and Harmonization Simultaneously

An Enhancement and Detection Method for a Glue Dispensing Image Based on the CycleGAN Model

Do Inpainting Yourself: Generative Facial Inpainting Guided by Exemplars