Quality Assessment for Tone-Mapped HDR Images Using Multi-Scale and Multi-Layer Information

Qin He,Dingquan Li,Tingting Jiang,Ming Jiang

DOI: https://doi.org/10.1109/ICMEW.2018.8551502

2018-10-23

Abstract:Tone mapping operators and multi-exposure fusion methods allow us to enjoy the informative contents of high dynamic range (HDR) images with standard dynamic range devices, but also introduce distortions into HDR contents. Therefore methods are needed to evaluate tone-mapped image quality. Due to the complexity of possible distortions in a tone-mapped image, information from different scales and different levels should be considered when predicting tone-mapped image quality. So we propose a new no-reference method of tone-mapped image quality assessment based on multi-scale and multi-layer features that are extracted from a pre-trained deep convolutional neural network model. After being aggregated, the extracted features are mapped to quality predictions by regression. The proposed method is tested on the largest public database for TMIQA and compared to existing no-reference methods. The experimental results show that the proposed method achieves better performance.

Multimedia

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to evaluate the quality of tone - mapped high - dynamic - range (HDR) images, especially in the absence of reference images. Specifically, the paper proposes a new no - reference (NR) image quality assessment method based on multi - scale and multi - level information to deal with the complex distortions that may occur in tone - mapped images.

### Problem Background

1. **HDR Images and Their Challenges**
- HDR images provide a wider range of luminance variation, but most devices can only display standard - dynamic - range (SDR) images.
- Through tone - mapping operators or multi - exposure fusion methods, HDR content can be displayed on SDR devices, but this will introduce complex distortions.

2. **Limitations of Existing Evaluation Methods**
- Subjective evaluation is accurate but time - consuming and complex.
- Objective evaluation methods need to consider multiple factors, such as luminance, contrast, and naturalness, and existing NR - IQA methods are insufficient in dealing with these complex distortions.

### Core Contributions of the Paper

1. **Multi - scale and Multi - level Feature Extraction**
- Extract information at different scales (local and global) to ensure that both details and overall structure are considered during evaluation.
- Use a pre - trained deep convolutional neural network (DCNN) model (such as ResNet - 50) to extract low - level, mid - level, and high - level features from different layers.

2. **Feature Aggregation and Regression Prediction**
- Aggregate the extracted features through mean and standard deviation pooling.
- Use partial least squares regression (PLSR) to map the aggregated features to quality prediction scores.

3. **Experimental Verification**
- A large number of experiments were carried out on the ESPL - LIVE HDR database, proving that this method is superior to existing NR - IQA methods in performance.

### Summary

This paper aims to develop a no - reference method that can effectively evaluate the quality of tone - mapped HDR images. By combining multi - scale and multi - level information, the accuracy and robustness of the evaluation are improved. The experimental results show that this method outperforms other no - reference image quality assessment methods on existing databases.