CalliPaint: Chinese Calligraphy Inpainting with Diffusion Model

Qisheng Liao,Zhinuo Wang,Muhammad Abdul-Mageed,Gus Xia

2023-12-04

Abstract:Chinese calligraphy can be viewed as a unique form of visual art. Recent advancements in computer vision hold significant potential for the future development of generative models in the realm of Chinese calligraphy. Nevertheless, methods of Chinese calligraphy inpainting, which can be effectively used in the art and education fields, remain relatively unexplored. In this paper, we introduce a new model that harnesses recent advancements in both Chinese calligraphy generation and image inpainting. We demonstrate that our proposed model CalliPaint can produce convincing Chinese calligraphy.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main objective of this paper is to address the problem of Chinese calligraphy inpainting. Specifically, the authors propose a new model called CalliPaint, which utilizes a diffusion model to restore existing Chinese calligraphy images. This method has broad application potential, especially in the fields of art and education. By combining the Calliffusion model and the RePaint scheduler, CalliPaint can perform high-quality restoration of missing parts in calligraphy images under given text conditions. Experimental results show that the model achieves high accuracy in both objective assessments and subjective evaluations, and the generated calligraphy works are difficult to distinguish from real ones, particularly when judged by individuals familiar with Chinese calligraphy. Additionally, the paper discusses potential future applications and challenges of this technology, such as the relatively slow inference speed.