Abstract:Image inpainting aims at restoring missing region of corrupted images, which has many applications such as image restoration and object removal. However, current GAN-based inpainting models fail to explicitly consider the semantic consistency between restored images and original images. Forexample, given a male image with image region of one eye missing, current models may restore it with a female eye. This is due to the ambiguity of GAN-based inpainting models: these models can generate many possible restorations given a missing region. To address this limitation, our key insight is that semantically interpretable information (such as attribute and segmentation information) of input images (with missing regions) can provide essential guidance for the inpainting process. Based on this insight, we propose a boosted GAN with semantically interpretable information for image inpainting that consists of an inpainting network and a discriminative network. The inpainting network utilizes two auxiliary pretrained networks to discover the attribute and segmentation information of input images and incorporates them into the inpainting process to provide explicit semantic-level guidance. The discriminative network adopts a multi-level design that can enforce regularizations not only on overall realness but also on attribute and segmentation consistency with the original images. Experimental results show that our proposed model can preserve consistency on both attribute and segmentation level, and significantly outperforms the state-of-the-art models.

Image Inpainting Based on Multi-Scale Stable-Field GAN

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Multi-Scale Generative Adversarial Networks Image Inpainting Algorithm*

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Multi-scale semantic image inpainting with residual learning and GAN.

A Context-Based Multi-Scale Discriminant Model for Natural Image Inpainting

Face Image Inpainting Based on Generative Adversarial Network

Single-shot Semantic Image Inpainting with Densely Connected Generative Networks

MSE-Net: Generative Image Inpainting with Multi-Scale Encoder

Context-Aware Semantic Inpainting

Boosted GAN with Semantically Interpretable Information for Image Inpainting

Image Fine-grained Inpainting.

Multi-scale patch-GAN with edge detection for image inpainting

Multistage Semantic‐aware Image Inpainting with Stacked Generator Networks

Generative Image Inpainting for Large-Scale Edge Area

Image Inpainting Based on Patch-GANs

Semantic Image Inpainting with Multi-Stage Feature Reasoning Generative Adversarial Network

A Progressive Image Inpainting Algorithm with a Mask Auto-update Branch

Image Inpainting Based on Contextual Coherent Attention GAN

Fine-grained image inpainting with scale-enhanced generative adversarial network

Multi-scale Generative Adversarial Network with Multi-Head External Attention for Image Inpainting