Abstract:Image aesthetic quality assessment has becoming an indispensable technique that facilitates a variety of image applications, e.g., photo retargeting and non-realistic rendering. Conventional approaches suffer from the following limitations: 1) the inefficiency of semantically describing images due to the inherent tag noise and incompletion, 2) the difficulty of accurately reflecting how humans actively perceive various regions inside each image, and 3) the challenge of incorporating the aesthetic experiences of multiple users. To solve these problems, we propose a novel semi-supervised deep active learning (SDAL) algorithm, which discovers how humans perceive semantically important regions from a large quantity of images partially assigned with contaminated tags. More specifically, as humans usually attend to the foreground objects before understanding them, we extract a succinct set of BING (binarized normed gradients) [60]-based object patches from each image. To simulate human visual perception, we propose SDAL which hierarchically learns human gaze shifting path (GSP) by sequentially linking semantically important object patches from each scenery. Noticeably, SDLA unifies the semantically important regions discovery and deep GSP feature learning into a principled framework, wherein only a small proportion of tagged images are adopted. Moreover, based on the sparsity penalty, SDLA can optimally abandon the noisy or redundant low-level image features. Finally, by leveraging the deeply-learned GSP features, a probabilistic model is developed for image aesthetics assessment, where the experience of multiple professional photographers can be encoded. Besides, auxiliary quality-related features can be conveniently integrated into our probabilistic model. Comprehensive experiments on a series of benchmark image sets have demonstrated the superiority of our method. As a byproduct, eye tracking experiments have shown that GSPs generated by our SDAL are about 93% consistent with real human gaze shifting paths.

Media Quality Assessment by Perceptual Gaze-Shift Patterns Discovery

Community-Aware Photo Quality Evaluation by Deeply Encoding Human Perception

A Biologically Inspired Automatic System for Media Quality Assessment

Human Visual Perception Based Image Quality Assessment for Video Prediction

Biologiclly Inspired Media Quality Modeling

Video Quality Assessment Combining with Human Visual Gaze Characteristics

Perception-Guided Multimodal Feature Fusion For Photo Aesthetics Assessment

Perceptual Quality Assessment of 360$^\circ$ Images Based on Generative Scanpath Representation

Image Quality Assessment: Exploring Regional Heterogeneity via Response of Adaptive Multiple Quality Factors in Dictionary Space

Quality assessment of perceptual color video based on a top-down framework and quaternion

Fusion of multichannel local and global structural cues for photo aesthetics evaluation.

Video quality assessment using content-partitioned approach

Quality Assessment for Video with Degradation Along Salient Trajectories

Image Quality Assessment Based on Matching Pursuit.

Image quality assessment based on matching pursuit

Deep Active Learning with Contaminated Tags for Image Aesthetics Assessment

A novel objective quality assessment method for perceptual video coding in conversational scenarios

No-reference Quality Assessment for Contrast-Distorted Images Based on Multifaceted Statistical Representation of Structure.

A Blind Video Quality Assessment Method Via Spatiotemporal Pyramid Attention

Learned Scanpaths Aid Blind Panoramic Video Quality Assessment

Quality Assessment of Free-viewpoint Videos by Quantifying the Elastic Changes of Multi-Scale Motion Trajectories