Abstract:We explore the question: "How much prior art knowledge is needed to create art?" To investigate this, we propose a text-to-image generation model trained without access to art-related content. We then introduce a simple yet effective method to learn an art adapter using only a few examples of selected artistic styles. Our experiments show that art generated using our method is perceived by users as comparable to art produced by models trained on large, art-rich datasets. Finally, through data attribution techniques, we illustrate how examples from both artistic and non-artistic datasets contributed to the creation of new artistic styles.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper explores a crucial question: **Is it necessary to rely on prior artistic knowledge to create artworks?** Specifically, the authors attempt to answer the following questions: 1. **Can a model be trained with a small number of art samples to enable it to have the ability to generate artworks?** 2. **Can a model that has hardly been exposed to art learn to imitate and generalize these styles by introducing a small number of art samples of specific styles?** To verify these questions, the authors propose a new method, that is, using the **Art - Free Diffusion Model** (diffusion model without artistic knowledge) to generate images. This model is first pre - trained on a dataset that contains almost no artistic content, and then fine - tuned by introducing a module named **Art Adapter** and using a small number of samples of specific artistic styles. ### Main contributions of the paper 1. **Development of the Art - Free SAM dataset**: This is a strictly - screened text - to - image dataset, ensuring that it contains almost no artistic elements. 2. **Proposing the Art Adapter method**: This is a simple and effective method that can inject artistic styles through a small number of art samples without changing the original model structure. 3. **Verification of the model's effectiveness**: Through user perception research, quantitative evaluation, and artist interviews, the potential of the Art - Free Diffusion Model in generating artworks has been proven. ### Method overview 1. **Constructing the Art - Free Diffusion Model**: - Use the Art - Free SAM dataset for pre - training to ensure that the model has not been exposed to artistic content. - The model architecture is based on the Latent Diffusion Model, including VAE encoder, UNet, and Text Encoder. 2. **Introducing Art Adapter**: - Through LoRA (Low - Rank Adaptation) technology, introduce low - rank adapters in the attention, linear, and convolutional layers of UNet. - Use a small number of samples of specific artistic styles for fine - tuning, enabling the model to learn and reproduce these styles. 3. **Experiment and evaluation**: - Conduct experiments on multiple benchmark datasets and compare the performance of different models. - Through user perception research and quantitative evaluation, verify the effectiveness of the Art - Free Diffusion Model. ### Conclusion The research shows that even in the case of hardly being exposed to art, by introducing a small number of samples of specific artistic styles, the model can still generate high - quality images with artistic styles. This provides a new perspective for future research, especially in terms of how to balance ethical issues and model capabilities. Through this method, the authors not only challenge the existing models' dependence on a large amount of artistic data but also provide a new way of thinking to solve the ethical problems in art generation.

Art-Free Generative Models: Art Creation Without Graphic Art Knowledge

Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge.

Generative Models for the Psychology of Art and Aesthetics

Generative Creativity: Adversarial Learning For Bionic Design

Creative and Diverse Artwork Generation Using Adversarial Networks

Human Aesthetic Preference-Based Large Text-to-Image Model Personalization: Kandinsky Generation as an Example

Inventing art styles with no artistic training data

Inspirational Adversarial Image Generation

RealtimeGen: an Intervenable AI Image Generation System for Commercial Digital Art Asset Creators

Collaborative Interactive Evolution of Art in the Latent Space of Deep Generative Models

GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models

Generative artificial intelligence, human creativity, and art

Content-Conditioned Generation of Stylized Free hand Sketches

Using Aesthetic Learning Model to Create Art.

Learning to Evaluate the Artness of AI-generated Images

A Framework and Dataset for Abstract Art Generation via CalligraphyGAN

Art and the science of generative AI: A deeper dive

GenerativeGI: creating generative art with genetic improvement

Generative AI and the History of Architecture

Rethinking Artistic Copyright Infringements in the Era of Text-to-Image Generative Models