Theme-Aware Aesthetic Distribution Prediction With Full-Resolution Photographs
Gengyun Jia,Peipei Li,Ran He
DOI: https://doi.org/10.1109/tnnls.2022.3151787
IF: 14.255
2022-01-01
IEEE Transactions on Neural Networks and Learning Systems
Abstract:Aesthetic quality assessment (AQA) is a challenging task due to complex aesthetic factors. Currently, it is common to conduct AQA using deep neural networks (DNNs) that require fixed-size inputs. The existing methods mainly transform images by resizing, cropping, and padding or use adaptive pooling to alternately capture the aesthetic features from fixed-size inputs. However, these transformations potentially damage aesthetic features. To address this issue, we propose a simple but effective method to accomplish full-resolution image AQA by combining image padding with region of image (RoM) pooling. Padding turns inputs into the same size. RoM pooling pools image features and discards extra padded features to eliminate the side effects of padding. In addition, the image aspect ratios are encoded and fused with visual features to remedy the shape information loss of RoM pooling. Furthermore, we observe that the same image may receive different aesthetic evaluations under different themes, which we call the theme criterion bias. Hence, a theme-aware model that uses theme information to guide model predictions is proposed. Finally, we design an attention-based feature fusion module to effectively use both the shape and theme information. Extensive experiments prove the effectiveness of the proposed method over state-of-the-art methods.
computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture