AI Art Neural Constellation: Revealing the Collective and Contrastive State of AI-Generated and Human Art

Faizan Farooq Khan,Diana Kim,Divyansh Jha,Youssef Mohamed,Hanna H Chang,Ahmed Elgammal,Luba Elliott,Mohamed Elhoseiny
2024-02-04
Abstract:Discovering the creative potentials of a random signal to various artistic expressions in aesthetic and conceptual richness is a ground for the recent success of generative machine learning as a way of art creation. To understand the new artistic medium better, we conduct a comprehensive analysis to position AI-generated art within the context of human art heritage. Our comparative analysis is based on an extensive dataset, dubbed ``ArtConstellation,'' consisting of annotations about art principles, likability, and emotions for 6,000 WikiArt and 3,200 AI-generated artworks. After training various state-of-the-art generative models, art samples are produced and compared with WikiArt data on the last hidden layer of a deep-CNN trained for style classification. We actively examined the various art principles to interpret the neural representations and used them to drive the comparative knowledge about human and AI-generated art. A key finding in the semantic analysis is that AI-generated artworks are visually related to the principle concepts for modern period art made in 1800-2000. In addition, through Out-Of-Distribution (OOD) and In-Distribution (ID) detection in CLIP space, we find that AI-generated artworks are ID to human art when they depict landscapes and geometric abstract figures, while detected as OOD when the machine art consists of deformed and twisted figures. We observe that machine-generated art is uniquely characterized by incomplete and reduced figuration. Lastly, we conducted a human survey about emotional experience. Color composition and familiar subjects are the key factors of likability and emotions in art appreciation. We propose our whole methodologies and collected dataset as our analytical framework to contrast human and AI-generated art, which we refer to as ``ArtNeuralConstellation''. Code is available at:
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The paper attempts to address the issue of understanding and comparing the differences between AI-generated artworks and human-created artworks. Specifically, the study aims to analyze and position AI-generated art within the context of human artistic heritage through the following aspects: 1. **Formal Analysis**: Conduct a formal analysis of AI-generated artworks based on Heinrich Wölfflin's art principles and other general art principles, and compare them with human artworks. 2. **Temporal Analysis**: Determine which period of human art AI-generated artworks are closest to in the time dimension. 3. **Emotional and Preference Analysis**: Understand whether AI-generated artworks can evoke similar emotional experiences as human art by investigating human emotional responses and preferences towards these works. 4. **Out-Of-Distribution (OOD) Detection**: Identify the features that make AI-generated artworks different from human artworks by detecting their distribution in the CLIP space. The study constructs an analytical framework named "ArtNeuralConstellation," using a large-scale dataset (including 6,000 WikiArt works and 3,200 AI-generated artworks) and various state-of-the-art generative models (such as StyleGAN, VQ-GAN, DDPM, etc.) for comprehensive comparative analysis. Ultimately, the research aims to provide guidance for the art and AI communities to help design and analyze the next generation of generative art.