Abstract:Image inpainting involves filling missing areas of a corrupted image. Despite impressive results have been achieved recently, restoring images with both vivid textures and reasonable structures remains a significant challenge. Previous methods have primarily addressed regular textures while disregarding holistic structures due to the limited receptive fields of Convolutional Neural Networks (CNNs). To this end, we study learning a Zero-initialized residual addition based Incremental Transformer on Structural priors (ZITS++), an improved model upon our conference work, ZITS (Dong et al. 2022). Specifically, given one corrupt image, we present the Transformer Structure Restorer (TSR) module to restore holistic structural priors at low image resolution, which are further upsampled by Simple Structure Upsampler (SSU) module to higher image resolution. To recover image texture details, we use the Fourier CNN Texture Restoration (FTR) module, which is strengthened by Fourier and large-kernel attention convolutions. Furthermore, to enhance the FTR, the upsampled structural priors from TSR are further processed by Structure Feature Encoder (SFE) and optimized with the Zero-initialized Residual Addition (ZeroRA) incrementally. Besides, a new masking positional encoding is proposed to encode the large irregular masks. Compared with ZITS, ZITS++ improves the FTR's stability and inpainting ability with several techniques. More importantly, we comprehensively explore the effects of various image priors for inpainting and investigate how to utilize them to address high-resolution image inpainting with extensive experiments. This investigation is orthogonal to most inpainting approaches and can thus significantly benefit the community.

Total Variation and Texture Synthesis Applied to Multi-Level Image Inpainting

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

Research of Image Restoration Algorithms

A new hybrid multi-start tabu search for finding hidden purchase decision strategies in WWW based on eye-movements

Sketch-Guided Texture-Based Image Inpainting

Research on image inpainting algorithm of improved total variation minimization method

Texture Image Denoising Algorithm Based on Structure Tensor and Total Variation

Highly Efficient Local Non-Texture Image Inpainting Based on Partial Differential Equation

A novel texture synthesis based algorithm for object removal in photographs

Curvature-Dependent Elastic Bending Total Variation Model for Image Inpainting with the SAV Algorithm

Exemplar-Based Image Inpainting Using Structure Consistent Patch Matching

Mathematical Models for Local Nontexture Inpaintings

Inpainting based on total variation

Standards-enabled Smart Grid for the future Energy Web

Simultaneous Inpainting of Image Structure and Texture with PDE

Delving Globally into Texture and Structure for Image Inpainting

An Adaptive Texture-Preserved Image Denoising Model

VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing

Image inpainting based on large displacement view images

FFTI: Image Inpainting Algorithm via Features Fusion and Two-Steps Inpainting

ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors.