ZiGAN: Fine-grained Chinese Calligraphy Font Generation via a Few-shot Style Transfer Approach

Qi Wen,Shuang Li,Bingfeng Han,Yi Yuan
DOI: https://doi.org/10.1145/3474085.3475225
2021-08-08
Abstract:Chinese character style transfer is a very challenging problem because of the complexity of the glyph shapes or underlying structures and large numbers of existed characters, when comparing with English letters. Moreover, the handwriting of calligraphy masters has a more irregular stroke and is difficult to obtain in real-world scenarios. Recently, several GAN-based methods have been proposed for font synthesis, but some of them require numerous reference data and the other part of them have cumbersome preprocessing steps to divide the character into different parts to be learned and transferred separately. In this paper, we propose a simple but powerful end-to-end Chinese calligraphy font generation framework ZiGAN, which does not require any manual operation or redundant preprocessing to generate fine-grained target-style characters with few-shot references. To be specific, a few paired samples from different character styles are leveraged to attain a fine-grained correlation between structures underlying different glyphs. To capture valuable style knowledge in target and strengthen the coarse-grained understanding of character content, we utilize multiple unpaired samples to align the feature distributions belonging to different character styles. By doing so, only a few target Chinese calligraphy characters are needed to generated expected style transferred characters. Experiments demonstrate that our method has a state-of-the-art generalization ability in few-shot Chinese character style transfer.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the fine - grained style transfer problem in Chinese calligraphy font generation. Specifically, the paper focuses on how to use a small number of samples (i.e., few - shot learning) to achieve high - quality Chinese calligraphy font style transfer. Chinese calligraphy style transfer is an extremely challenging task because the glyph structures of Chinese characters are complex and there are a large number of Chinese characters. In addition, calligraphers' handwritten works are difficult to obtain in real - life scenarios, especially those with irregular strokes. Although existing GAN - based methods can generate fonts, they usually require a large amount of reference data or complex pre - processing steps, which are often not feasible in practical applications. To address these challenges, the authors propose a new framework named ZiGAN. ZiGAN aims to generate fine - grained target - style characters from a small number of target - style samples without any manual operations or redundant pre - processing steps. ZiGAN uses a small number of paired samples (several pairs of samples of different character styles) to obtain the fine - grained correlations between different glyph structures, and uses multiple unpaired samples to align the feature distributions belonging to different character styles, in order to capture valuable knowledge in the target style and enhance the coarse - grained understanding of character content. Experimental results show that ZiGAN has state - of - the - art generalization ability in few - shot Chinese character style transfer.