Abstract:Recently, deep learning-based image inpainting methods have made great strides in reconstructing damaged regions. However, these methods often struggle to produce satisfactory results when dealing with missing images with large holes, leading to distortions in the structure and blurring of textures. To address these problems, we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms. The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details. This method divides the inpainting task into two phases: edge prediction and image inpainting. Specifically, in the edge prediction phase, a transformer architecture is designed to combine axial attention with standard self-attention. This design enhances the extraction capability of global structural features and location awareness. It also balances the complexity of self-attention operations, resulting in accurate prediction of the edge structure in the defective region. In the image inpainting phase, a multi-scale fusion attention module is introduced. This module makes full use of multi-level distant features and enhances local pixel continuity, thereby significantly improving the quality of image inpainting. To evaluate the performance of our method, comparative experiments are conducted on several datasets, including CelebA, Places2, and Facade. Quantitative experiments show that our method outperforms the other mainstream methods. Specifically, it improves Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) by 1.141~3.234 db and 0.083~0.235, respectively. Moreover, it reduces Learning Perceptual Image Patch Similarity (LPIPS) and Mean Absolute Error (MAE) by 0.0347~0.1753 and 0.0104~0.0402, respectively. Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details. Furthermore, our model exhibits impressive performance in terms of the number of parameters, memory cost, and testing time.

Image inpainting exploiting global prior refined weighted low-rank representation

Non-Local Image Inpainting Using Low-Rank Matrix Completion: Non-Local Image Inpainting Using Low-Rank Matrix Completion

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

Efficient Completion for Corrupted Low-Rank Images Via Alternating Direction Method

Image Inpainting Using Non-Convex Low Rank Decomposition and Multidirectional Search

Global optimization based image inpainting andits implementation on GPU

Restoration Guarantee of Image Inpainting via Low Rank Patch Matrix Completion

Robust image restoration via adaptive low-rank approximation and joint kernel regression.

Group-Based Sparse Representation Based on Lp-Norm Minimization for Image Inpainting

Image Inpainting Algorithm Based on Tensor Decomposition and Weighted Nuclear Norm

Image inpainting by low-rank tensor decomposition and multidirectional search

Nonlocal Patches based Gaussian Mixture Model for Image Inpainting

Image Inpainting Via Weighted Sparse Non-negative Matrix Factorization

Group Sparsity Residual Constraint with Non-Local Priors for Image Restoration

Assessing Image Inpainting Via Re-Inpainting Self-Consistency Evaluation

An Edge Information and Mask Shrinking Based Image Inpainting Approach

Non-Local Meets Global: an Iterative Paradigm for Hyperspectral Image Restoration

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering

Image inpainting based on sparse representation using self-similar joint sparse coding

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism