Abstract:Recently, due to the strength of deep convolutional neural networks (CNN), many CNN-based image quality assessment (IQA) models have been studied. However, previous CNN-based IQA models likely have yet to utilize the characteristics of the human visual system (HVS) fully for IQA problems when they simply entrust everything to the CNN, expecting it to learn from a training dataset. However, in this paper, we propose a novel saliency-channel attention residual network based on the just-noticeable-difference (JND) concept for full-reference image quality assessments (FR-IQA). It is referred to as JND-SalCAR and shows significant improvements in large IQA datasets with various types of distortion. The proposed JND-SalCAR effectively learns how to incorporate human psychophysical characteristics, such as visual saliency and JND, into image quality predictions. In the proposed network, a SalCAR block is devised so that perceptually important features can be extracted with the help of saliency-based spatial attention and channel attention schemes. In addition, a saliency map serves as a guideline for predicting a patch weight map in order to afford stable training of end-to-end optimization for the JND-SalCAR. To the best of our knowledge, our work presents the first HVS-inspired trainable FR-IQA network that considers both visual saliency and the JND characteristics of the HVS. When the visual saliency map and the JND probability map are explicitly given as priors, they can be usefully combined to predict IQA scores rated by humans more precisely, eventually leading to performance improvements and faster convergence. The experimental results show that the proposed JND-SalCAR significantly outperforms all recent state-of-the-art FR-IQA methods on large IQA datasets in terms of the Spearman rank order coefficient (SRCC) and the Pearson linear correlation coefficient (PLCC).

Deep Saliency Quality Assessment Network with Joint Metric

SCVS: Blind Image Quality Assessment Based on Spatial Correlation and Visual Saliency

A Novel Just-Noticeable-Difference-based Saliency-Channel Attention Residual Network for Full-Reference Image Quality Predictions

Multi Information Fusion Network for Saliency Quality Assessment.

DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection

Bridging the Gap Between Saliency Prediction and Image Quality Assessment

SGDNet

Deep supervised visual saliency model addressing low-level features

Blind Surveillance Image Quality Assessment Via Deep Neural Network Combined with the Visual Saliency

Deeply-Supervised Recurrent Convolutional Neural Network for Saliency Detection.

Point Cloud Quality Assessment using 3D Saliency Maps

Ransp: Ranking Attention Network for Saliency Prediction on Omnidirectional Images

Activity guided multi-scales collaboration based on scaled-CNN for saliency prediction

Visual Saliency Based Blind Image Quality Assessment Via Convolutional Neural Network

Saliency Prediction on Omnidirectional Images with Brain-Like Shallow Neural Network.

UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

Deep Saliency with Encoded Low level Distance Map and High Level Features

DMRA: Depth-Induced Multi-Scale Recurrent Attention Network for RGB-D Saliency Detection

Saliency Detection Based on Multiple-Level Feature Learning

Spatial Attention-based Non-reference Perceptual Quality Prediction Network for Omnidirectional Images

What Do Deep Saliency Models Learn about Visual Attention?