Abstract:In this work, we introduce a challenging image restoration task, referred to as SuperInpaint, which aims to reconstruct missing regions in low-resolution images and generate completed images with arbitrarily higher resolutions. We have found that this task cannot be effectively addressed by stacking state-of-the-art super-resolution and image inpainting methods as they amplify each other's flaws, leading to noticeable artifacts. To overcome these limitations, we propose the detail-enhanced attentional implicit representation (DEAR) that can achieve SuperInpaint with a single model, resulting in high-quality completed images with arbitrary resolutions. Specifically, we use a deep convolutional network to extract the latent embedding of an input image and then enhance the high-frequency components of the latent embedding via an adaptive high-pass filter. This leads to detail-enhanced semantic embedding. We further feed the semantic embedding into an unmask-attentional module that suppresses embeddings from ineffective masked pixels. Additionally, we extract a pixel-wise importance map that indicates which pixels should be used for image reconstruction. Given the coordinates of a pixel we want to reconstruct, we first collect its neighboring pixels in the input image and extract their detail-enhanced semantic embeddings, unmask-attentional semantic embeddings, importance values, and spatial distances to the desired pixel. Then, we feed all the above terms into an implicit representation and generate the color of the specified pixel. To evaluate our method, we extend three existing datasets for this new task and build 18 meaningful baselines using SOTA inpainting and super-resolution methods. Extensive experimental results demonstrate that our method outperforms all existing methods by a significant margin on four widely used metrics.

ConvSRGAN: super-resolution inpainting of traditional Chinese paintings

Sgrgan: sketch-guided restoration for traditional Chinese landscape paintings

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

SRInpaintor: When Super-Resolution Meets Transformer for Image Inpainting

Learning Structral Coherence Via Generative Adversarial Network for Single Image Super-Resolution

Chinese Ancient Paintings Inpainting Based on Edge Guidance and Multi-Scale Residual Blocks

SRAGAN: Saliency Regularized and Attended Generative Adversarial Network for Chinese Ink-wash Painting Generation

An Ancient Chinese Painting Restoration Method Based on Improved Generative Adversarial Network

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Traditional landscape painting and art image restoration methods based on structural information guidance

Hierarchical Painter: Chinese Landscape Painting Restoration with Fine-Grained Styles.

SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting

Texture Hallucination for Large-Factor Painting Super-Resolution

SR-Inpaint: A General Deep Learning Framework for High Resolution Image Inpainting

AM-ESRGAN: Super-Resolution Reconstruction of Ancient Murals Based on Attention Mechanism and Multi-Level Residual Network

Better Visual Image Super-Resolution with Laplacian Pyramid of Generative Adversarial Networks

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Data augmentation techniques based on deep learning for Chinese paintings

DSRGAN: Detail Prior-Assisted Perceptual Single Image Super-Resolution via Generative Adversarial Networks