Multi-task Feature Enhancement Network for No-Reference Image Quality Assessment

Li Yu

2024-11-12

Abstract:Due to the scarcity of labeled samples in Image Quality Assessment (IQA) datasets, numerous recent studies have proposed multi-task based strategies, which explore feature information from other tasks or domains to boost the IQA task. Nevertheless, multi-task strategies based No-Reference Image Quality Assessment (NR-IQA) methods encounter several challenges. First, existing methods have not explicitly exploited texture details, which significantly influence the image quality. Second, multi-task methods conventionally integrate features through simple operations such as addition or concatenation, thereby diminishing the network's capacity to accurately represent distorted features. To tackle these challenges, we introduce a novel multi-task NR-IQA framework. Our framework consists of three key components: a high-frequency extraction network, a quality estimation network, and a distortion-aware network. The high-frequency extraction network is designed to guide the model's focus towards high-frequency information, which is highly related to the texture details. Meanwhile, the distortion-aware network extracts distortion-related features to distinguish different distortion types. To effectively integrate features from different tasks, a feature fusion module is developed based on an attention mechanism. Empirical results from five standard IQA databases confirm that our method not only achieves high performance but also exhibits robust generalization ability.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

This paper attempts to address several key issues in No-Reference Image Quality Assessment (NR-IQA):

1. **Insufficient explicit use of texture details**: Existing methods fail to explicitly utilize texture details, which significantly impact image quality.
2. **Simplicity of multi-task feature fusion**: Current multi-task methods typically fuse features through simple operations (such as addition or concatenation), which weakens the network's ability to accurately represent distortion features.
3. **Small dataset size and insufficient samples**: The current NR-IQA datasets are small in scale and lack sufficient training samples, leading to poor performance of the trained models in practical applications.

To address these issues, the authors propose a new multi-task no-reference image quality assessment framework, which includes three key components:

1. **High-Frequency Extraction Network (HFEN)**: Designed to guide the model to focus on high-frequency information, which is highly related to texture details.
2. **Quality Estimation Network (QEN)**: As the main task, responsible for predicting the quality score of the image.
3. **Distortion-Aware Network (DAN)**: Extracts distortion-related features to distinguish different types of distortions.

To effectively fuse features from different tasks, the authors developed an attention-based Feature Fusion Module (FFM). Additionally, DAN is pre-trained using a contrastive learning method to enhance its generalization ability to unknown distortion types.

Experimental results show that this method not only achieves high performance on 5 standard IQA databases but also demonstrates strong generalization ability.