Abstract:Deep learning approaches for Image Aesthetics Assessment (IAA) have shown promising results in recent years, but the internal mechanisms of these models remain unclear. Previous studies have demonstrated that image aesthetics can be predicted using semantic features, such as pre-trained object classification features. However, these semantic features are learned implicitly, and therefore, previous works have not elucidated what the semantic features are representing. In this work, we aim to create a more transparent deep learning framework for IAA by introducing explainable semantic features. To achieve this, we propose Tag-based Content Descriptors (TCDs), where each value in a TCD describes the relevance of an image to a human-readable tag that refers to a specific type of image content. This allows us to build IAA models from explicit descriptions of image contents. We first propose the explicit matching process to produce TCDs that adopt predefined tags to describe image contents. We show that a simple MLP-based IAA model with TCDs only based on predefined tags can achieve an SRCC of 0.767, which is comparable to most state-of-the-art methods. However, predefined tags may not be sufficient to describe all possible image contents that the model may encounter. Therefore, we further propose the implicit matching process to describe image contents that cannot be described by predefined tags. By integrating components obtained from the implicit matching process into TCDs, the IAA model further achieves an SRCC of 0.817, which significantly outperforms existing IAA methods. Both the explicit matching process and the implicit matching process are realized by the proposed TCD generator. To evaluate the performance of the proposed TCD generator in matching images with predefined tags, we also labeled 5101 images with photography-related tags to form a validation set. And experimental results show that the proposed TCD generator can meaningfully assign photography-related tags to images.

Theme-Aware Visual Attribute Reasoning for Image Aesthetics Assessment

UMAAF: Unveiling Aesthetics via Multifarious Attributes of Images

Learning Image Aesthetic Assessment from Object-level Visual Components

Image Aesthetics Assessment With Attribute-Assisted Multimodal Memory Network

Theme-Aware Aesthetic Distribution Prediction With Full-Resolution Photographs

Inkthetics: A Comprehensive Computational Model for Aesthetic Evaluation of Chinese Ink Paintings.

Improving Image Aesthetic Assessment via Multiple Image Joint Learning

Context-aware Attention Network for Predicting Image Aesthetic Subjectivity

Image Aesthetics Assessment With Emotion-Aware Multibranch Network

Towards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Method

Semantic and style based multiple reference learning for artistic and general image aesthetic assessment

Towards Transparent Deep Image Aesthetics Assessment with Tag-based Content Descriptors.

Neural aesthetic image reviewer

UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark

Image Aesthetics Assessment via Learnable Queries

Learning Image Aesthetic Subjectivity from Attribute-Aware Relational Reasoning Network

Personalized Image Aesthetics Assessment Via Multi-Attribute Interactive Reasoning

Distilling Knowledge from Object Classification to Aesthetics Assessment

Aesthetic Visual Question Answering of Photographs

Image Aesthetic Assessment Assisted by Attributes Through Adversarial Learning.

Aesthetically Relevant Image Captioning