Chinese Ancient Paintings Inpainting Based on Edge Guidance and Multi-Scale Residual Blocks

Zengguo Sun,Yanyan Lei,Xiaojun Wu

DOI: https://doi.org/10.3390/electronics13071212

IF: 2.9

2024-03-27

Electronics

Abstract:Chinese paintings have great cultural and artistic significance and are known for their delicate lines and rich textures. Unfortunately, many ancient paintings have been damaged due to historical and natural factors. The deep learning methods that are successful in restoring natural images cannot be applied to the inpainting of ancient paintings. Thus, we propose a model named Edge-MSGAN for inpainting Chinese ancient paintings based on edge guidance and multi-scale residual blocks. The Edge-MSGAN utilizes edge images to direct the completion network in order to generate entire ancient paintings. It then applies the multi-branch color correction network to adjust the colors. Furthermore, the model uses multi-scale channel attention residual blocks to learn the semantic features of ancient paintings at various levels. At the same time, by using polarized self-attention, the model can improve its concentration on significant structures, edges, and details, which leads to paintings that possess clear lines and intricate details. Finally, we have created a dataset for ancient paintings inpainting, and have conducted experiments in order to evaluate the model's performance. After comparing the proposed model with state-of-the-art models from qualitative and quantitative aspects, it was found that our model is better at inpainting the texture, edge, and color of ancient paintings. Therefore, our model achieved maximum PSNR and SSIM values of 34.7127 and 0.9280 respectively, and minimum MSE and LPIPS values of 0.0006 and 0.0495, respectively.

engineering, electrical & electronic,computer science, information systems,physics, applied

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to address the issues in the restoration of Chinese ancient paintings, particularly the damage and fading caused by historical and natural factors. Although existing deep learning methods have achieved success in natural image restoration, these methods do not perform well in the restoration of ancient paintings. Specifically, the paper proposes a model based on edge guidance and multi-scale residual blocks (Edge-MSGAN) for the restoration of Chinese ancient paintings.

#### Main Issues:
1. **Insufficient Feature Extraction**: Existing methods do not sufficiently extract features when dealing with ancient paintings that have rich textures and complex structures.
2. **Loss of Details**: Detail information is easily lost during the restoration process.
3. **Inaccurate Color Control**: Existing methods tend to produce blurriness and semantic loss when restoring large damaged areas and struggle to achieve precise color control, which may lead to color deviations in the output image.

#### Solutions:
- **Edge Guidance**: Use the edge images of the original ancient paintings to guide the restoration network, ensuring that the restored content aligns with the edges of the original image.
- **Multi-Scale Residual Blocks (MSRB)**: Utilize multi-scale residual blocks to extract detailed features, expand the perceptual range, and better preserve the rich textures and features of ancient paintings.
- **Multi-Branch Color Correction Network**: Construct a multi-branch color correction network to effectively adjust the colors of the restored areas, reduce color deviations, and fully utilize feature information at different levels, making the colors of the restored ancient paintings rich and close to the original.

Through the above methods, Edge-MSGAN can better preserve textures, edges, and colors during the restoration of ancient paintings. The final experimental results show that its performance is superior to existing models.