Visual Interaction Perceptual Network for Blind Image Quality Assessment
Xiaoqi Wang,Jian Xiong,Weisi Lin
DOI: https://doi.org/10.1109/tmm.2023.3243683
IF: 7.3
2023-01-01
IEEE Transactions on Multimedia
Abstract:In observing images, the perception of the human visual system (HVS) is affected by both image contents and distortions. Obviously, the visual quality of the same image varies under different distortion types and intensities. Furthermore, the visual masking effects reveal that image content and distortion have a visual interaction, where the HVS presents different visibility of the identical distortion for different image contents. Based upon this, we propose a visual interaction perceptual network that can perceive both content and distortion of an image. The proposed model consists of three sub-modules: content perception module (CPM), distortion perception module (DPM), and visual interaction module (VIM). However, the subjective quality score cannot guide the model to explicitly learn the feature representations of image content and distortion. Thus, we perform a two-stage training procedure. In the first stage, we obtain CPM and DPM, where semantic features are extracted to recognize the image content in CPM, and distortion features are extracted to capture the image distortion type and intensity in DPM. In the second stage, the VIM is applied to model the interaction between semantic and distortion features, and the final predicted quality score is given by a fully connected layer. Experimental results demonstrate that the proposed method can achieve state-of-the-art performance on multiple benchmark databases, e.g., CSIQ, TID2013, KADID-10K, and KonIQ-10 K.