Abstract:In a typical image inpainting task, the location and shape of the damaged or masked area is often random and irregular. The vanilla convolutions widely used in learning-based inpainting models treat all spatial features as valid and share parameters across regions, making it difficult for them to cope with those irregular damages, and models tend to produce inpainting results with color discrepancy and blurriness. In this paper, we propose a novel Context Adaptive Network (CANet) to address this issue. The main idea of the proposed CANet is able to generate different weights depending on the miscellaneous input, which may help to complement images with multiple broken forms in a flexible way. Specifically, the proposed CANet has two novel context adaptive modules, namely, the context adaptive block (CAB) and the cross-scale contextual attention (CSCA), which utilize attention mechanisms to cope with diverse content breakdowns. The proposed CAB, during the forward propagation, uses an adaptive term to determine the importance between adaptive term and convolution kernel, so as to dynamically balance features based on the degree of breakage (confidence level or soft mask), and the overall calculation is formulated as a classic convolution implementation with an additional attention term to describe local structure. Besides, the proposed CSCA, not only takes advantage of the contextual attention module, but also considers cross-scale information transfer to generate reasonable features for damaged areas, thus alleviating the inefficiency of the long-range modeling capability of convolutional neural networks. Qualitative and quantitative experiments show that our method performs better than state-of-the-arts, producing clearer, more coherent and visually plausible inpainting results. The code can be found at github.com/dengyecode/CANet_image_inpainting.

Image Inpainting Based on Cross-Hierarchy Global and Local Aware Network.

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Image Inpainting Based on Contextual Coherent Attention GAN

Image Inpainting Network Based on Multi‐level Attention Mechanism

Deep Image Inpainting with Enhanced Normalization and Contextual Attention

Hourglass Attention Network for Image Inpainting.

Image Inpainting with Multi-scale Consistent Features

Pyramid-VAE-GAN: Transferring Hierarchical Latent Variables for Image Inpainting

Context Adaptive Network for Image Inpainting.

Image Inpainting Based Multi-scale Gated Convolution and Attention

Multi-scale Gated Inpainting Network with Patch-Wise Spacial Attention.

Globally Consistent Image Inpainting Based on WGAN-GP Network Optimization

High-Resolution Image Inpainting Based On Multi-Scale Neural Network

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

GLCSA-Net: Global–local Constraints-Based Spectral Adaptive Network for Hyperspectral Image Inpainting

Image Inpainting Using Wasserstein Generative Adversarial Network

Multi-scale Generative Adversarial Inpainting Network Based on Cross-Layer Attention Transfer Mechanism.

LRGAN:A computational lightweight image inpainting neural network

A Deep Network Architecture for Image Inpainting

Ahff-Net: Adaptive Hierarchical Feature Fusion Network For Image Inpainting

Attentional coarse-and-fine generative adversarial networks for image inpainting