Refining the Unseen: Self-supervised Two -stream Feature Extraction for Image Quality Assessment

Yiwei Lou,Yanyuan Chen,Dexuan Xu,Doudou Zhou,Yongzhi Cao,Hanpin Wang,Yu Huang

DOI: https://doi.org/10.1109/ICDM58522.2023.00147

2023-01-01

Abstract:The inadequacy of labeled datasets for image quality assessment has led to the development and popularity of selfsupervised approaches. However, most existing self-supervised methods primarily focus on content and fidelity features extracted with convolutional neural networks, overlooking the crucial importance of structural features in quality assessment. To address this problem, we present a novel self-supervised two-stream feature extraction and representation approach. In our approach, the first stream leverages a contrastive learning framework to extract image fidelity features, while the second stream emphasizes structural features by incorporating an attention mechanism. This innovative combination results in a comprehensive feature representation for quality assessment. Moreover, our proposed method facilitates transfer learning, allowing the pre -trained Iwo stream model in the source domain to be seamlessly applied to target domains for quality regression. This compatibility with transfer learning enhances the adaptability and generalization of the model. Extensive experiments are carried out on three synthetic distortion datasets to validate the effectiveness of our approach. The results demonstrate that our work not only competes with state-of-the-art self-supervised methods but also outperforms some supervised approaches.