The Performance of Two CNN Methods in Artworks Aesthetic Feature Recognition

Jipeng Gao,Haolin Zhou,Yicheng Zhang

DOI: https://doi.org/10.1145/3383972.3383974

2020-01-01

Abstract:As the technology of computer vision become more advanced, it allows us to be able to classify the styles, genres and artists of artworks with the help of computer. However, never can we know how the convolutional neural networks(CNNs) extract and recognize those aesthetic features like objective or subjective elements. We apply two CNNs: VGG19 and ResNet-50 on different artworks. We compare the results from these networks to understand how these two networks work when they recognize the underlying features such as aesthetic feature. The dataset is obtained from.the best art-work on the world.. We selected five subsets from this dataset which have the most paintings and belong to five different artists. Meanwhile, these five artists have five different styles. We get 86.72% accuracy on the validation set by VGG19, while we get 82.31% accuracy on the validation set by ResNet-50. We then use many approaches such as kernel visualization, grad-cam heat-map, confusion matrix and style transfer to explore how those two convolutional neural networks extract the underlying features. By analyzing the results we can conclude that the CNNs actually has the ability to extract and learn the underlying features such as aesthetic features. We discover that different CNNs have different tendencies to extract the specific underlying features. Specifically, VGG19 prefers to extract subjective feature but ResNet-50 prefers to learn objective feature.