HySim: An Efficient Hybrid Similarity Measure for Patch Matching in Image Inpainting

Saad Noufel,Nadir Maaroufi,Mehdi Najib,Mohamed Bakhouya

2024-03-21

Abstract:Inpainting, for filling missing image regions, is a crucial task in various applications, such as medical imaging and remote sensing. Trending data-driven approaches efficiency, for image inpainting, often requires extensive data preprocessing. In this sense, there is still a need for model-driven approaches in case of application constrained with data availability and quality, especially for those related for time series forecasting using image inpainting techniques. This paper proposes an improved modeldriven approach relying on patch-based techniques. Our approach deviates from the standard Sum of Squared Differences (SSD) similarity measure by introducing a Hybrid Similarity (HySim), which combines both strengths of Chebychev and Minkowski distances. This hybridization enhances patch selection, leading to high-quality inpainting results with reduced mismatch errors. Experimental results proved the effectiveness of our approach against other model-driven techniques, such as diffusion or patch-based approaches, showcasing its effectiveness in achieving visually pleasing restorations.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the patch - matching problem in inpainting, especially that the traditional methods in dealing with complex textures and structures are prone to introduce cumulative errors, leading to a decline in the inpainting quality. Specifically, the existing exemplar - based approaches rely on the Sum of Squared Differences (SSD) similarity measure when selecting the most similar patches, and this method may lead to inappropriate patch selection, thereby generating cumulative errors and affecting the final inpainting effect.

To solve this problem, the author proposes a new Hybrid Similarity (HySim) measure, which combines the advantages of Chebyshev distance and Minkowski distance. Through this hybrid method, more accurate selection of similar patches can be achieved, cumulative errors can be reduced, and thus the quality of the inpainting results can be improved.

### Main contributions of the paper:
1. **Research on exemplar - based methods**: Verify their effectiveness through experimental tests.
2. **Distinguish between the concepts of similarity and distance**: Emphasize the inherent duality between the two.
3. **Introduce the HySim measure**: Combine the advantages of Chebyshev and Minkowski distances to improve patch - matching performance.

### Formula representation:
- **Chebyshev distance**: \[ d_{\text{Chebyshev}}(\Psi_p, \Psi_q) = \max_i | \Psi_p(i) - \Psi_q(i) | \]
- **Minkowski distance**: \[ d_{\text{Minkowski}}(\Psi_p, \Psi_q; P) = \left( \sum_i | \Psi_p(i) - \Psi_q(i) |^P \right)^{\frac{1}{P}} \]
- **HySim hybrid similarity measure**: \[ d_{P,\infty}^{\alpha,\beta} = \alpha \max | \Psi_p - \Psi_q | + \beta \left( \sum_q | \Psi_p - \Psi_q |^P \right) \]

In this way, HySim can consider the similarity of patches simultaneously on local and global scales, thereby improving the quality of image inpainting and reducing cumulative errors.

### Summary:
This paper aims to improve the exemplar - based image inpainting methods by introducing the HySim measure, especially in the performance when dealing with complex textures and structures. The experimental results show that HySim can significantly improve the inpainting effect, reduce cumulative errors, and provide higher - quality inpainting results.