Abstract:Image inpainting that completes large free-form missing regions in images is a promising yet challenging task. State-of-the-art approaches have achieved significant progress by taking advantage of generative adversarial networks (GAN). However, these approaches can suffer from generating distorted structures and blurry textures in high-resolution images (e.g., 512×512). The challenges mainly drive from (1) image content reasoning from distant contexts, and (2) fine-grained texture synthesis for a large missing region. To overcome these two challenges, we propose an enhanced GAN-based model, named Aggregated COntextual-Transformation GAN (AOT-GAN), for high-resolution image inpainting. Specifically, to enhance context reasoning, we construct the generator of AOT-GAN by stacking multiple layers of a proposed AOT block. The AOT blocks aggregate contextual transformations from various receptive fields, allowing to capture both informative distant image contexts and rich patterns of interest for context reasoning. For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task. Such a training objective forces the discriminator to distinguish the detailed appearances of real and synthesized patches, and in turn facilitates the generator to synthesize clear textures. Extensive comparisons on Places2, the most challenging benchmark with 1.8 million high-resolution images of 365 complex scenes, show that our model outperforms the state-of-the-art. A user study including more than 30 subjects further validates the superiority of AOT-GAN. We further evaluate the proposed AOT-GAN in practical applications, e.g., logo removal, face editing, and object removal. Results show that our model achieves promising completions in the real world. We release codes and models in https://github.com/researchmm/AOT-GAN-for-Inpainting.

A lightweight image inpainting model for removing unwanted objects from residential real estate's indoor scenes

Face Image Inpainting Based on Generative Adversarial Network

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Aggregated Contextual Transformations for High-Resolution Image Inpainting

A Progressive Image Inpainting Algorithm with a Mask Auto-update Branch

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

GAN-Based Object Removal in High-Resolution Satellite Images

Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks

Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

Semantics-Guided Object Removal for Facial Images: with Broad Applicability and Robust Style Preservation

Towards Realistic Landmark-Guided Facial Video Inpainting Based on GANs

A Method for Face Image Inpainting Based on Autoencoder and Generative Adversarial Network.

RSINet: Inpainting Remotely Sensed Images Using Triple GAN Framework

Image Inpainting Based on Contextual Coherent Attention GAN

Inst-Inpaint: Instructing to Remove Objects with Diffusion Models

Efficient texture-aware multi-GAN for image inpainting

A three-stage GAN model based on edge and color prediction for image outpainting

Boosted GAN with Semantically Interpretable Information for Image Inpainting

DAM-GAN : Image Inpainting using Dynamic Attention Map based on Fake Texture Detection

Real-Time Image Inpainting using PatchMatch Based Two-Generator Adversarial Networks with Optimized Edge Loss Function.

MagicEraser: Erasing Any Objects via Semantics-Aware Control