Generalized W-Net: Arbitrary-style Chinese Character Synthesization

Haochuan Jiang,Guanyu Yang,Fei Cheng,Kaizhu Huang
2024-06-11
Abstract:Synthesizing Chinese characters with consistent style using few stylized examples is challenging. Existing models struggle to generate arbitrary style characters with limited examples. In this paper, we propose the Generalized W-Net, a novel class of W-shaped architectures that addresses this. By incorporating Adaptive Instance Normalization and introducing multi-content, our approach can synthesize Chinese characters in any desired style, even with limited examples. It handles seen and unseen styles during training and can generate new character contents. Experimental results demonstrate the effectiveness of our approach.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **Generate Chinese characters with arbitrary styles based on a few - shot (few - sample examples), while maintaining the consistency of character content**. Specifically, existing models have difficulty in generating diverse stylized characters when using limited examples, especially when dealing with complex Chinese characters. The Generalized W - Net proposed in this paper aims to overcome these problems by introducing new architectures and improved techniques. ### Problem Background 1. **Complex Character Structure**: Chinese characters are composed of multiple strokes and radicals, with complex and diverse structures. 2. **Difficult Style Transfer**: Existing deep - generation models have poor performance when dealing with handwritten characters, especially when there are only a few examples. 3. **Separation of Style and Content**: A method is required to separate the content and style of characters and be able to flexibly switch between different styles. ### Solutions To solve the above problems, the paper proposes the following improvements: 1. **Extension of the W - Net Architecture**: - **Multiple Content Prototypes**: Introduce multiple content prototypes so that the model can combine different style features. - **Adaptive Instance Normalization (AdaIN)**: Used to enhance the effect of style transfer. - **Residual Blocks and Dense Blocks**: Improve the performance of the model. 2. **Feature Mixer**: - Use the AdaIN technique to normalize the extracted content features, making them better adapt to different style features. - Introduce multiple normalization methods in the feature mixer, such as Batch Normalization (BN), Instance Normalization (IN), and Layer Normalization (LN), to improve the robustness and generalization ability of the model. 3. **Adversarial Training**: - Train based on the Wasserstein GAN with Gradient Penalty (W - GAN - GP) framework to ensure that the generated characters are visually close to real characters. - Introduce multiple loss functions, including adversarial loss, pixel reconstruction loss, and perceptual loss, to optimize the generation effect. ### Experimental Results The experimental results show that the Generalized W - Net can not only generate high - quality stylized characters with a few examples, but also handle unseen styles and generate new character content. In addition, the model also demonstrates cross - language generalization ability and can generate characters of other East Asian languages (such as Korean and Japanese). ### Summary By proposing the Generalized W - Net, this paper solves the problem of existing models in generating diverse stylized Chinese characters in the few - sample case, providing new ideas and technical means for future research.