Abstract:Image inpainting refers to the task of generating a complete, natural image based on a partially revealed reference image. Recently, many research interests have been focused on addressing this problem using fixed diffusion models. These approaches typically directly replace the revealed region of the intermediate or final generated images with that of the reference image or its variants. However, since the unrevealed regions are not directly modified to match the context, it results in incoherence between revealed and unrevealed regions. To address the incoherence problem, a small number of methods introduce a rigorous Bayesian framework, but they tend to introduce mismatches between the generated and the reference images due to the approximation errors in computing the posterior distributions. In this paper, we propose COPAINT, which can coherently inpaint the whole image without introducing mismatches. COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions, but approximates the posterior distribution in a way that allows the errors to gradually drop to zero throughout the denoising steps, thus strongly penalizing any mismatches with the reference image. Our experiments verify that COPAINT can outperform the existing diffusion-based methods under both objective and subjective metrics. The codes are available at <a class="link-external link-https" href="https://github.com/UCSB-NLP-Chang/CoPaint/" rel="external noopener nofollow">this https URL</a>.

A Local Nontexture Image Inpainting and Denoising Based on Nonlinear Pdes

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

A Local Nontexture Image Inpainting and Denoising Based on Anisotropic Diffusion Equation

Non-Local Image Inpainting Using Low-Rank Matrix Completion: Non-Local Image Inpainting Using Low-Rank Matrix Completion

Inpainting based on total variation

Inpainting for Remotely Sensed Images With a Multichannel Nonlocal Total Variation Model

A Decoupled Method for Image Inpainting with Patch-Based Low Rank Regulariztion.

Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering

Highly Efficient Local Non-Texture Image Inpainting Based on Partial Differential Equation

Partial Differential Equation Inpainting Method Based on Image Characteristics.

Image Denoising Algorithms Based on Laplacian Operator and Image Inpainting

A New Algorithm for Image Inpainting Based on the Navier-Stokes Equation

Free-Form Image Inpainting with Separable Gate Encoder-Decoder Network

A Novel Non-local Image Inpainting Algorithm: Fractal-Based Image Inpainting

Orthogonal-directional Forward Diffusion Image Inpainting and Denoising Model

Mathematical Models for Local Nontexture Inpaintings

Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models

Wavelet Inpainting by Nonlocal Total Variation

Nonlocal Patches based Gaussian Mixture Model for Image Inpainting

A universal variational framework for sparsity-based image inpainting

An image inpainting technique based on nonparametric kernel estimation