Intelligent-paint: a Chinese Painting Process Generation Method Based on Vision Transformer

Zunfu Wang,Fang Liu,Zhixiong Liu,Changjuan Ran,Mohan Zhang

DOI: https://doi.org/10.1007/s00530-024-01316-w

IF: 3.9

2024-01-01

Multimedia Systems

Abstract:The generation of painting steps can help people understand how artistic works are created and assist beginners in learning through copying. However, this task faces significant challenges: achieving the generation of clear and plausible intermediate painting steps while maintaining consistency with the real painting process. Existing related research mainly focuses on generating painting steps for oil painting using brush stroke rendering methods. However, such approaches often result in significant discrepancies between the generated process and the real painting process, making it challenging to reflect the principles and techniques of painting accurately, and they are not applicable to Chinese painting. To better address the issue of generating painting steps for Chinese painting, we propose “Intelligent-paint”. First, considering the unique painting principles of Chinese painting, we interpret the painting process as a mapping from the final artwork to a series of intermediate painting stages. To ensure the quality of the generated intermediate stages, we use a generator based on Vision Transformer (Vit) for this mapping process. We enhance the image generation quality by adversarial learning with a real/fake discriminator. In addition, to capture the characteristics of Chinese painting, such as void and brush strokes, we employ void loss constraint and brush stroke loss constraint to ensure consistency with the features of Chinese painting. To ensure the coherence between the generated painting sequence and the real painting process, we employ a sequence discriminator to constrain the generated painting sequence. Expert evaluations and quantitative assessments indicate that our method outperforms existing approaches. Through ablation experiments and applicability evaluations, our method demonstrates strong rationality and applicability, providing significant assistance to beginners in learning Chinese painting.