Abstract:In this work, we introduce a challenging image restoration task, referred to as SuperInpaint, which aims to reconstruct missing regions in low-resolution images and generate completed images with arbitrarily higher resolutions. We have found that this task cannot be effectively addressed by stacking state-of-the-art super-resolution and image inpainting methods as they amplify each other's flaws, leading to noticeable artifacts. To overcome these limitations, we propose the detail-enhanced attentional implicit representation (DEAR) that can achieve SuperInpaint with a single model, resulting in high-quality completed images with arbitrary resolutions. Specifically, we use a deep convolutional network to extract the latent embedding of an input image and then enhance the high-frequency components of the latent embedding via an adaptive high-pass filter. This leads to detail-enhanced semantic embedding. We further feed the semantic embedding into an unmask-attentional module that suppresses embeddings from ineffective masked pixels. Additionally, we extract a pixel-wise importance map that indicates which pixels should be used for image reconstruction. Given the coordinates of a pixel we want to reconstruct, we first collect its neighboring pixels in the input image and extract their detail-enhanced semantic embeddings, unmask-attentional semantic embeddings, importance values, and spatial distances to the desired pixel. Then, we feed all the above terms into an implicit representation and generate the color of the specified pixel. To evaluate our method, we extend three existing datasets for this new task and build 18 meaningful baselines using SOTA inpainting and super-resolution methods. Extensive experimental results demonstrate that our method outperforms all existing methods by a significant margin on four widely used metrics.

Towards Stable and Faithful Inpainting

Single-Mask Inpainting for Voxel-Based Neural Radiance Fields

A Progressive Image Inpainting Algorithm with a Mask Auto-update Branch

SafePaint: Anti-forensic Image Inpainting with Domain Adaptation

Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling

Assessing Image Inpainting Via Re-Inpainting Self-Consistency Evaluation

An Adaptive Iterative Inpainting Method with More Information Exploration

Diverse Image Inpainting with Disentangled Uncertainty

Learning Prior Feature and Attention Enhanced Image Inpainting

Coherent and Multi-modality Image Inpainting via Latent Space Optimization

PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control

An Edge Information and Mask Shrinking Based Image Inpainting Approach

Deformable Dynamic Sampling and Dynamic Predictable Mask Mining for Image Inpainting

SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting

Inpainting with Separable Mask Update Convolution Network

UniPaint: Unified Space-time Video Inpainting via Mixture-of-Experts

Region-wise Generative Adversarial ImageInpainting for Large Missing Areas

ARIN: Adaptive Resampling and Instance Normalization for Robust Blind Inpainting of Dunhuang Cave Paintings

Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal