Abstract:Image inpainting infers the missing areas of a corrupted image according to the information of the undamaged part. Many existing image inpainting methods can generate plausible inpainted results from damaged images with the fast-developed deep-learning technology. However, they still suffer from over-smoothed textures or textural distortion in the cases of complex textural details or large damaged areas. To restore textures at a fine-grained level, we propose an image inpainting method based on a hierarchical VQ-VAE with a vector credibility mechanism. It first trains the hierarchical VQ-VAE with ground truth images to update two codebooks and to obtain two corresponding vector collections containing information on ground truth images. The two vector collections are fed to a decoder to generate the corresponding high-fidelity outputs. An encoder then is trained with the corresponding damaged image. It generates vector collections approximating the ground truth by the help of the prior knowledge provided by the codebooks. After that, the two vector collections pass through the decoder from the hierarchical VQ-VAE to produce the inpainted results. In addition, we apply a vector credibility mechanism to promote vector collections from damaged images and approximate vector collections from ground truth images. To further improve the inpainting result, we apply a refinement network, which uses residual blocks with different dilation rates to acquire both global information and local textural details. Extensive experiments conducted on several datasets demonstrate that our method outperforms the state-of-the-art ones.

Variational Inpainting Model Based on Alternate Iteration and Its Split Bregman Algorithm

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

A Variational Approach to Medical Image Inpainting Based on Mumford-Shah Model

A Graph-based Iterative Algorithm for Inverse Problems with Local and Non-local Regularizations

Patch-based Variational Image Approximation

A universal variational framework for sparsity-based image inpainting

A Progressive Image Inpainting Algorithm with a Mask Auto-update Branch

Split Bregman Iteration Algorithm for Image Deblurring Using Fourth-Order Total Bounded Variation Regularization Model

Curvature-Dependent Elastic Bending Total Variation Model for Image Inpainting with the SAV Algorithm

A Class of Fractional-Order Multi-Scale Variational Models and Alternating Projection Algorithm for Image Denoising

Inpainting based on total variation

A Novel Variational Image Model: Towards a Unified Approach to Image Editing

A variational level set model based on additive decomposition for segmenting noisy images with intensity inhomogeneity

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

A new hybrid multi-start tabu search for finding hidden purchase decision strategies in WWW based on eye-movements

A Total Fractional-Order Variation Model for Image Restoration with Non-homogeneous Boundary Conditions and its Numerical Solution

The Navier-Stokes-Voight Model for Image Inpainting

Total variation image reconstruction algorithm based on non-convex function

An Image Inpainting Approach Based on the Poisson Equation

A Hybrid Inpainting Model Combining Diffusion and Enhanced Exemplar Methods

Hierarchical Vector-Quantized Variational Autoencoder and Vector Credibility Mechanism for High-Quality Image Inpainting