Abstract:With the advancement of deep learning, artificial intelligence (AI) has made many breakthroughs in recent years and achieved superhuman performance in various tasks such as object detection, reading comprehension, and video games. Generative Modeling, such as various Generative Adversarial Networks (GAN) models, has been applied to generate paintings and music. Research in Natural Language Processing (NLP) also had a leap forward in 2018 since the release of the pre-trained contextual neural language models such as BERT and recently released GPT3. Despite the exciting AI applications aforementioned, AI is still significantly lagging behind humans in creativity, which is often considered the ultimate moonshot for AI. Our work is inspired by Chinese calligraphy, which is a unique form of visual art where the character itself is an aesthetic painting. We also draw inspirations from paintings of the Abstract Expressionist movement in the 1940s and 1950s, such as the work by American painter Franz Kline. In this paper, we present a creative framework based on Conditional Generative Adversarial Networks and Contextual Neural Language Model to generate abstract artworks that have intrinsic meaning and aesthetic value, which is different from the existing work, such as image captioning and text-to-image generation, where the texts are the descriptions of the images. In addition, we have publicly released a Chinese calligraphy image dataset and demonstrate our framework using a prototype system and a user study.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the application of artificial intelligence technology in the field of artistic creation, especially the generation of abstract artworks with intrinsic meaning and aesthetic value by combining Chinese calligraphy and Abstract Expressionist painting styles. Specifically, the authors aim to develop a creative framework based on Conditional Generative Adversarial Networks (CGAN) and Contextual Neural Language Model to generate abstract artworks related to specific texts (such as dish names). This goal is different from existing image captioning or text - to - image generation tasks, where text is usually used as a direct description of the image, while the goal of this paper is to generate artworks that can reflect the intrinsic meaning of the text. The key points in the paper include: - **Dataset construction**: The authors collected 138,499 pictures of Chinese characters written by 19 Chinese calligraphers, covering 7,328 different characters, and selected 1,000 characters with at least 25 different pictures for each character for training. - **Text mapping algorithm**: Use the pre - trained BERT model to generate the embedding representations of the input text (i.e., Chinese dish names) and the 1,000 characters, then calculate the similarity between them, and select the top five most similar characters as control conditions. - **Generative model**: Use these five characters as conditions to generate new, never - before - seen characters through CalligraphyGAN, and these new characters incorporate the shape features of the five original characters. - **Artistic processing**: Denoise and perform style transfer on the generated images to make them closer to the oil painting style, and provide additional style transfer options based on the works of famous painters. - **Prototype system and evaluation**: Cooperate with a restaurant in Shanghai to build a prototype system that allows users to customize generation parameters and collect feedback through user studies to evaluate the performance and user experience of the system. In conclusion, this paper explores a novel application of artificial intelligence creativity by combining deep learning and traditional Chinese art elements, aiming to enhance the diversity and interactivity of artistic creation.

A Framework and Dataset for Abstract Art Generation via CalligraphyGAN

Self-Adaptive Computational Aesthetic Evaluation of Chinese Ink Paintings Based on Deep Learning

Creative and Diverse Artwork Generation Using Adversarial Networks

SmartPaint: a Co-Creative Drawing System Based on Generative Adversarial Networks

Style Generation in Robot Calligraphy with Deep Generative Adversarial Networks

Innovative Application of Chinese Landscape Culture Painting Elements in AI Art Generation

RoDAL: style generation in robot calligraphy with deep adversarial learning

On GANs Art in Context of Artificial Intelligence Art

RealtimeGen: an Intervenable AI Image Generation System for Commercial Digital Art Asset Creators

Brushstrokes of Tomorrow: Exploring the Art of AI

Auto-Encoder Guided GAN for Chinese Calligraphy Synthesis

Continuation of Famous Art with AI: A Conditional Adversarial Network Inpainting Approach

Automating Generative Deep Learning for Artistic Purposes: Challenges and Opportunities

Artistic characterization of AI painting based on generative adversarial networks

Colour and Brush Stroke Pattern Recognition in Abstract Art using Modified Deep Convolutional Generative Adversarial Networks

End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks

Generative artificial intelligence, human creativity, and art

Everyone Can Be Picasso? A Computational Framework into the Myth of Human versus AI Painting

AI Painting: An Aesthetic Painting Generation System

The Development and Impact of AI-Generated Content in Contemporary Painting

AI Art Neural Constellation: Revealing the Collective and Contrastive State of AI-Generated and Human Art