Abstract:In the field of objective image quality assessment (IQA), the Spearman's $\rho$ and Kendall's $\tau$ are two most popular rank correlation indicators, which straightforwardly assign uniform weight to all quality levels and assume each pair of images are sortable. They are successful for measuring the average accuracy of an IQA metric in ranking multiple processed images. However, two important perceptual properties are ignored by them as well. Firstly, the sorting accuracy (SA) of high quality images are usually more important than the poor quality ones in many real world applications, where only the top-ranked images would be pushed to the users. Secondly, due to the subjective uncertainty in making judgement, two perceptually similar images are usually hardly sortable, whose ranks do not contribute to the evaluation of an IQA metric. To more accurately compare different IQA algorithms, we explore a perceptually weighted rank correlation indicator in this paper, which rewards the capability of correctly ranking high quality images, and suppresses the attention towards insensitive rank mistakes. More specifically, we focus on activating `valid' pairwise comparison towards image quality, whose difference exceeds a given sensory threshold (ST). Meanwhile, each image pair is assigned an unique weight, which is determined by both the quality level and rank deviation. By modifying the perception threshold, we can illustrate the sorting accuracy with a more sophisticated SA-ST curve, rather than a single rank correlation coefficient. The proposed indicator offers a new insight for interpreting visual perception behaviors. Furthermore, the applicability of our indicator is validated in recommending robust IQA metrics for both the degraded and enhanced image data.

Pairwise comparison and rank learning for image quality assessment

A New Objective Assessment Method for Image Coding Quality

Comparison of Image Quality Models for Optimization of Image Processing Systems

Active Sampling Exploiting Reliable Informativeness for Subjective Image Quality Assessment Based on Pairwise Comparison

Ranking Consistent Rate: New Evaluation Criterion on Pairwise Subjective Experiments

Learning to Rank for Blind Image Quality Assessment

Pairwise Comparisons Are All You Need

A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment

Comparison of Full-Reference Image Quality Models for Optimization of Image Processing Systems

Quality Assessment for Comparing Image Enhancement Algorithms

Strategy for Boosting Pair Comparison and Improving Quality Assessment Accuracy

Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare

Content-Diverse Comparisons improve IQA

Multiscale contrast similarity deviation: An effective and efficient index for perceptual image quality assessment

dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs

Image Quality Assessment Based on Multi-Order Visual Comparison

Contrast and visual saliency similarity-induced index for assessing image quality

Intensity-Sensitive Similarity Indexes for Image Quality Assessment

Multi-task rank learning for image quality assessment

A Soft-ranked Index Fusion Framework with Saliency Weighting for Image Quality Assessment

Image Quality Assessment Based on Inter-Patch and Intra-Patch Similarity.