MTUP: Multi-Task CNN with Uncertainty and Probability for No-Reference Image Quality Assessment

Weifeng Shi,Junhao Chen,Bingrui Song,Yuzhen Niu
DOI: https://doi.org/10.1145/3488933.3488940
2021-01-01
Abstract:Current CNN-based image quality assessment (IQA) metrics treat all image patches equally. We observed that when the distortions do not appear or not noticeable in the image patch, its predicted quality score is usually away from the subjective score. In this paper, we argue that not all image patches equally matter for the IQA task and present a multi-task convolutional neural network (CNN) model with uncertainty and probability for no-reference IQA (MTUP). We also present a probability-aware average (PAA) method for pooling the image quality-score from the scores of the image patches. Because the loss functions of image quality-score prediction and distortion-type identification tasks have quantities with different units and scales, we further use a multi-task learning method that automatically weights the loss functions of multiple tasks with task uncertainty to solve the challenge of relative weights tune. Experimental results show that the proposed MTUP model achieved a superior performance as compared with state-of-the-art NR-IQA metrics.
What problem does this paper attempt to address?