Abstract:Recent image inpainting methods have achieved remarkable improvements by using generative adversarial networks (GAN). Most of them have been designed to produce plausible results from high-level semantic features using only high-resolution (HR) supervision. However, because abundant details are lost in large holes, it is difficult to simultaneously synthesize details while preserving structural coherence in HR space. Besides, the correlations between the inside and outside of the missing region play a critical role in transferring relevant known information to generate semantic-coherent textures, especially in patch matching-based methods. In this work, we present SRInpaintor which inherits the merits of super-resolution (SR) and transformer for high-fidelity image inpainting. The SRInpaintor starts from global structure reasoning with low-resolution (LR) input and progressively refines the local textures in HR space, constituting a multi-stage framework with SR supervision. The bottom stage recovers coarse SR results that provide structural information as an appearance prior, and is combined with the higher-resolution corrupted image at the next stage to render available textures for the missing region. Such a design can analyse the image from LR to HR with the increase of stages, enabling coarse-to-fine information propagation and detail refinement. In addition, we propose a hierarchical transformer (HieFormer) to model the long-term correlations between distant contexts and holes. By embedding it into a compact latent space in a cross-scale manner, we can ensure reliable relevant texture transformation and robust appearance consistency. Experimental results demonstrate the superiority of our method compared with recent state-of-the-art methods. Code will be available on https://github.com/lifengshiwo/SRInpaintor.

SLTFILL: Spatial and Light Transformer for Multi-Reference Image Inpainting

Thermal Infrared Image Inpainting Via Edge-Aware Guidance

3DFill:Reference-guided Image Inpainting by Self-supervised 3D Image Alignment

RealFill: Reference-Driven Generation for Authentic Image Completion

GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding

LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Model

TransInpaint: Transformer-based Image Inpainting with Context Adaptation

TransRef: Multi-Scale Reference Embedding Transformer for Reference-Guided Image Inpainting

Fill in the ____ (a Diffusion-based Image Inpainting Pipeline)

Bridging partial-gated convolution with transformer for smooth-variation image inpainting

Interactive Separation Network for Image Inpainting

Exemplar-based Image Inpainting Using Adaptive Two-Stage Structure-Tensor Based Priority Function and Nonlocal Filtering

SRInpaintor: When Super-Resolution Meets Transformer for Image Inpainting

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

GRIG: Few-Shot Generative Residual Image Inpainting

Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap

ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors.

StructureFlow: Image Inpainting Via Structure-aware Appearance Flow

FaithFill: Faithful Inpainting for Object Completion Using a Single Reference Image

FSTT: Flow-Guided Spatial Temporal Transformer for Deep Video Inpainting