Old scratched photos restoration with enhancement and colorization
Sheng Wang,Xin Ding,Wei Zhang
DOI: https://doi.org/10.1007/s11042-024-19595-w
IF: 2.577
2024-06-25
Multimedia Tools and Applications
Abstract:Although the current deep learning-based old photos restoration work can repair scratches and structural defects, the results often have the problems of incomplete scratch detection and poor repair quality. In order to solve these problems synthetically, in this paper, we use the way of modeling the degradation information of the scratched areas and generate contents to fill them. We proposed a framework to restore old scratched photos, in which we fully exploit and preserve the available information from the deteriorated photos, enabling us to accomplish image quality restoration and colorization with utmost finesse based on this invaluable data. Our proposed framework consists of two modules: a scratch removal module for old photos based on a Fine-tune and Guided the diffusion model, an Image enhancement and colorization module. To make full use of scratches feature information, we introduce a scratch distribution extraction (ScDE) module to learn the scratch model as Gaussian noise distributions. Through the fine-tuning the path of reverse denoising, it will eventually move towards the data distribution without scratches. Therefore, our proposed method can efficiently remove scratches from degraded old photos while maintaining the original styles of scratch-free regions. At the same time, most old photos after scanning have the problem of low resolution and blurriness, the efficacy of our image quality enhancement and Colorization strategy in resolving this issue is commendable. The comprehensive experimental results show that our image enhancement and colorization module can render the structural features of the image clearer and more realistic. Furthermore, we construct a large-scale old-photo-scratch-dataset (ScraOld) involved various scene classes with 703 pairs of scratched photos and their ground-truth. Experimental results demonstrate that our proposed method achieve the state-of-the-art (SOTA) performance in terms of PSNR, SSIM, LPIPS, FID metrics. The subjective quality is significantly higher than all comparison methods.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering