Abstract:Image inpainting aims to restore the missing regions of corrupted images and make the recovery result identical to the originally complete image, which is different from the common generative task emphasizing the naturalness or realism of generated images. Nevertheless, existing works usually regard it as a pure generation problem and employ cutting-edge deep generative techniques to address it. The generative networks can fill the main missing parts with realistic contents but usually distort the local structures or introduce obvious artifacts. In this paper, for the first time, we formulate image inpainting as a mix of two problems, predictive filtering and deep generation. Predictive filtering is good at preserving local structures and removing artifacts but falls short to complete the large missing regions. The deep generative network can fill the numerous missing pixels based on the understanding of the whole scene but hardly restores the details identical to the original ones. To make use of their respective advantages, we propose the joint predictive filtering and generative network (JPGNet) that contains three branches: predictive filtering & uncertainty network (PFUNet), deep generative network, and uncertainty-aware fusion network (UAFNet). The PFUNet can adaptively predict pixel-wise kernels for filtering-based inpainting according to the input image and output an uncertainty map. This map indicates the pixels should be processed by filtering or generative networks, which is further fed to the UAFNet for a smart combination between filtering and generative results. Note that, our method as a novel inpainting framework can benefit any existing generation-based methods. We validate our method on three public datasets, Dunhuang, Places2, and CelebA, and demonstrate that our method can enhance three state-of-the-art generative methods significantly with slightly extra time costs.

Facial image inpainting using attention-based multi-level generative network

Face Image Inpainting Based on Generative Adversarial Network

Towards Interactive Facial Image Inpainting by Text or Exemplar Image.

A Method for Face Image Inpainting Based on Autoencoder and Generative Adversarial Network.

Attentional coarse-and-fine generative adversarial networks for image inpainting

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Unsupervised masked face inpainting based on contrastive learning and attention mechanism

Face Image Inpainting Using Attribute Guided

Image Inpainting Using Multi-Scale Feature Joint Attention Model

Masked Face Inpainting Through Residual Attention UNet

Face inpainting based on GAN by facial prediction and fusion as guidance information

High-Resolution Image Inpainting Based On Multi-Scale Neural Network

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Multi-scale Generative Adversarial Network with Multi-Head External Attention for Image Inpainting

Reference-guided face inpainting with reference attention network

Multi-scale Generative Adversarial Inpainting Network Based on Cross-Layer Attention Transfer Mechanism.

MFMAM: Image inpainting via multi-scale feature module with attention module

JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting

Multi-scale Generative Model for Image Completion

Irregular mask image inpainting based on progressive generative adversarial networks

Image Inpainting Based on Multi-Level Feature Aggregation Network for Future Internet