Abstract:In a typical image inpainting task, the location and shape of the damaged or masked area is often random and irregular. The vanilla convolutions widely used in learning-based inpainting models treat all spatial features as valid and share parameters across regions, making it difficult for them to cope with those irregular damages, and models tend to produce inpainting results with color discrepancy and blurriness. In this paper, we propose a novel Context Adaptive Network (CANet) to address this issue. The main idea of the proposed CANet is able to generate different weights depending on the miscellaneous input, which may help to complement images with multiple broken forms in a flexible way. Specifically, the proposed CANet has two novel context adaptive modules, namely, the context adaptive block (CAB) and the cross-scale contextual attention (CSCA), which utilize attention mechanisms to cope with diverse content breakdowns. The proposed CAB, during the forward propagation, uses an adaptive term to determine the importance between adaptive term and convolution kernel, so as to dynamically balance features based on the degree of breakage (confidence level or soft mask), and the overall calculation is formulated as a classic convolution implementation with an additional attention term to describe local structure. Besides, the proposed CSCA, not only takes advantage of the contextual attention module, but also considers cross-scale information transfer to generate reasonable features for damaged areas, thus alleviating the inefficiency of the long-range modeling capability of convolutional neural networks. Qualitative and quantitative experiments show that our method performs better than state-of-the-arts, producing clearer, more coherent and visually plausible inpainting results. The code can be found at github.com/dengyecode/CANet_image_inpainting.

Learning Contextual Transformer Network for Image Inpainting

Context Adaptive Network for Image Inpainting.

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

The Improved Image Inpainting Algorithm Via Encoder and Similarity Constraint

TransInpaint: Transformer-based Image Inpainting with Context Adaptation

ITrans: generative image inpainting with transformers

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Interactive Separation Network for Image Inpainting

A Context-Based Multi-Scale Discriminant Model for Natural Image Inpainting

CTNet: hybrid architecture based on CNN and transformer for image inpainting detection

CTFCD: Channel Transformer Based on Full Convolutional Decoder for Single Image Deraining

HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention

Image Inpainting Based on Contextual Coherent Attention GAN

T-former: an Efficient Transformer for Image Inpainting

Inpainting with Sketch Reconstruction and Comprehensive Feature Selection

Generative Image Inpainting with Residual Texture Prior and Cross-Layer Contextual Attention

HT-Net: hierarchical context-attention transformer network for medical ct image segmentation

Sparse self-attention transformer for image inpainting

Image Inpainting with Contrastive Relation Network

Hourglass Attention Network for Image Inpainting.