Abstract:Image inpainting refers to the task of generating a complete, natural image based on a partially revealed reference image. Recently, many research interests have been focused on addressing this problem using fixed diffusion models. These approaches typically directly replace the revealed region of the intermediate or final generated images with that of the reference image or its variants. However, since the unrevealed regions are not directly modified to match the context, it results in incoherence between revealed and unrevealed regions. To address the incoherence problem, a small number of methods introduce a rigorous Bayesian framework, but they tend to introduce mismatches between the generated and the reference images due to the approximation errors in computing the posterior distributions. In this paper, we propose COPAINT, which can coherently inpaint the whole image without introducing mismatches. COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions, but approximates the posterior distribution in a way that allows the errors to gradually drop to zero throughout the denoising steps, thus strongly penalizing any mismatches with the reference image. Our experiments verify that COPAINT can outperform the existing diffusion-based methods under both objective and subjective metrics. The codes are available at <a class="link-external link-https" href="https://github.com/UCSB-NLP-Chang/CoPaint/" rel="external noopener nofollow">this https URL</a>.

A Hybrid Inpainting Model Combining Diffusion and Enhanced Exemplar Methods

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

Image Inpainting With Improved Exemplar-Based Approach

A PDE Constraint Adaptive-Exemplar Based Image Inpainting Model

An Exemplar Inpainting by Curvature-driven Method

Exemplar-Based Image Inpainting Using Structure Consistent Patch Matching

A Novel Hybrid Image Inpainting Model

Exemplar-Based Image Inpainting with Collaborative Filtering

Fill in the ____ (a Diffusion-based Image Inpainting Pipeline)

Exemplar-based Image Inpainting Using Adaptive Two-Stage Structure-Tensor Based Priority Function and Nonlocal Filtering

Sketch-guided Image Inpainting with Partial Discrete Diffusion Process

Hybrid Inpainting Algorithm Based on Sparse Representation and Fast Inpainting Method

Image Inpainting Based on Exemplar and Sparse Representation

Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models

Multi‐stage Image Inpainting Using Improved Partial Convolutions

An Edge Information and Mask Shrinking Based Image Inpainting Approach

Exemplar-Based Image Inpainting Using a Modified Priority Definition

An Adaptive Iterative Inpainting Method with More Information Exploration

Sparse Inpainting with Smoothed Particle Hydrodynamics

Fast Convergent Image Inpainting Method Based on Bscb Model

Two Fast Algorithms of Image Inpainting