Abstract:Deep learning approaches for Image Aesthetics Assessment (IAA) have shown promising results in recent years, but the internal mechanisms of these models remain unclear. Previous studies have demonstrated that image aesthetics can be predicted using semantic features, such as pre-trained object classification features. However, these semantic features are learned implicitly, and therefore, previous works have not elucidated what the semantic features are representing. In this work, we aim to create a more transparent deep learning framework for IAA by introducing explainable semantic features. To achieve this, we propose Tag-based Content Descriptors (TCDs), where each value in a TCD describes the relevance of an image to a human-readable tag that refers to a specific type of image content. This allows us to build IAA models from explicit descriptions of image contents. We first propose the explicit matching process to produce TCDs that adopt predefined tags to describe image contents. We show that a simple MLP-based IAA model with TCDs only based on predefined tags can achieve an SRCC of 0.767, which is comparable to most state-of-the-art methods. However, predefined tags may not be sufficient to describe all possible image contents that the model may encounter. Therefore, we further propose the implicit matching process to describe image contents that cannot be described by predefined tags. By integrating components obtained from the implicit matching process into TCDs, the IAA model further achieves an SRCC of 0.817, which significantly outperforms existing IAA methods. Both the explicit matching process and the implicit matching process are realized by the proposed TCD generator. To evaluate the performance of the proposed TCD generator in matching images with predefined tags, we also labeled 5101 images with photography-related tags to form a validation set. And experimental results show that the proposed TCD generator can meaningfully assign photography-related tags to images.

Photo Quality Assessment with DCNN that Understands Image Well.

Inkthetics: A Comprehensive Computational Model for Aesthetic Evaluation of Chinese Ink Paintings.

Self-Adaptive Computational Aesthetic Evaluation of Chinese Ink Paintings Based on Deep Learning

Community-Aware Photo Quality Evaluation by Deeply Encoding Human Perception

Image aesthetic evaluation using paralleled deep convolution neural network

Image Aesthetics Assessment Using Composite Features from off-the-Shelf Deep Models

Learning Photography Aesthetics with Deep CNNs

Deep Supervised Dictionary Learning for No-Reference Image Quality Assessment

Photo Aesthetics Ranking Network with Attributes and Content Adaptation

A-Lamp: Adaptive Layout-Aware Multi-patch Deep Convolutional Neural Network for Photo Aesthetic Assessment

Automatical Enhancement and Denoising of Extremely Low-light Images.

Deep Optimization model for Screen Content Image Quality Assessment using Neural Networks

Image Quality Assessment Guided Deep Neural Networks Training

A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping

Towards Transparent Deep Image Aesthetics Assessment with Tag-based Content Descriptors.

Matching Low-Quality Photo to DSLR-Quality with Deep Convolutional Networks

Brain-Inspired Deep Networks for Image Aesthetics Assessment

Blind Visual Quality Assessment for Image Super-Resolution by Convolutional Neural Network

DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator

Two-channel Deep Recursive Multi-Scale Network Based on Multi-Attention for No-Reference Image Quality Assessment

Multi-Task Convolutional Neural Network for Image Aesthetic Assessment