Smell and Emotion: Recognising emotions in smell-related artworks

Vishal Patoliya,Mathias Zinnen,Andreas Maier,Vincent Christlein

2024-07-05

Abstract:Emotions and smell are underrepresented in digital art history. In this exploratory work, we show that recognising emotions from smell-related artworks is technically feasible but has room for improvement. Using style transfer and hyperparameter optimization we achieve a minor performance boost and open up the field for future extensions.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to explore the technical feasibility of achieving individual-level emotion recognition in artworks related to scents. Specifically, the objectives of the study are:

1. **Addressing the cross-domain issue**: Due to the domain gap between artworks and photographs, emotion recognition neural networks trained on photographs perform poorly when applied to artworks.
2. **Application of emotion recognition in artworks**: Although existing datasets like WikiArt emotions and ArtEMis have cataloged viewers' emotional responses to artworks, the automatic recognition of emotions depicted in the characters of artworks has not been fully studied. This paper attempts to fill this gap.
3. **Multisensory and emotion analysis**: By focusing on scent-related artworks, the authors hope to pave the way for future research that can link the extracted emotions to previously identified olfactory references, thereby broadening the research perspective of computational humanities.

In summary, the core issue of this paper is to explore how to effectively recognize individual emotions in artworks and overcome the challenges posed by the domain differences between art and photography.