Abstract:Objective video quality assessment is of great importance in a variety of video processing applications. Most existing video quality metrics either focus primarily on capturing spatial artifacts in the video signal, or are designed to assess only grayscale video thereby ignoring important chrominance information. In this paper, on the basis of the top-down visual analysis of cognitive understanding and video features, we propose and develop a novel full-reference perceptual video assessment technique that accepts visual information inputs in the form of a quaternion consisting of contour, color and temporal information. Because of the more important role of chrominance information in the “border-to-surface” mechanism at early stages of cognitive visual processing, our new metric takes into account the chrominance information rather than the luminance information utilized in conventional video quality assessment. Our perceptual quaternion model employs singular value decomposition (SVD) and utilizes the human visual psychological features for SVD block weighting to better reflect perceptual focus and interest. Our major contributions include: a new perceptual quaternion that takes chrominance as one spatial feature, and temporal information to model motion or changes across adjacent frames; a three-level video quality measure to reflect visual psychology; and the two weighting methods based on entropy and frame correlation. Our experimental validation on the video quality experts’ group (VQEG) Phase I FR-TV test dataset demonstrated that our new assessment metric outperforms PSNR, SSIM, PVQM (P8) and has high correlation with perceived video quality.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
What problem does this paper attempt to address?
This paper aims to solve several key problems in video quality assessment, especially for the perceptual quality assessment of color videos. Most of the existing video quality measurement methods mainly focus on capturing spatial artifacts in video signals or are only designed to evaluate grayscale videos, thus ignoring important chrominance information. This paper proposes a full - reference perceptual video assessment technique based on a top - down framework and quaternions, with the following specific objectives:
1. **Introducing chrominance information**: Different from traditional video quality assessment methods that mainly use luminance information, the new measurement method proposed in this paper considers chrominance information, which plays a more important role in the "boundary - to - surface" mechanism of early cognitive visual processing.
2. **Modeling spatio - temporal information**: In order to more comprehensively reflect the quality of videos, this paper proposes a new perceptual quaternion model. This model not only contains contour, color and time information, but also uses singular value decomposition (SVD) to adjust block weights according to human visual psychological characteristics, so as to better reflect the perceptual focus and interest.
3. **Multi - level video quality measurement**: This paper develops a three - layer (i.e., block - level, frame - level and video - level) perceptual color video quality assessment system. This system uses the perceptual quaternion and its SVD as a single measurement and is suitable for various types of distortion.
4. **Improving assessment accuracy**: Experimental verification shows that the new assessment metric proposed in this paper performs better than PSNR, SSIM and PVQM (P8) on the VQEG Phase I FR - TV test data set and has a high correlation with the perceptual video quality.
### Formula Summary
- **Quaternion Definition**:
\[
q = a+bi + cj+dk
\]
where \(a, b, c, d\) are real numbers, and \(i, j, k\) satisfy the following rules:
\[
i^{2}=j^{2}=k^{2}=- 1,\quad i\cdot j = k,\quad j\cdot k = i,\quad k\cdot i = j,\quad j\cdot i=-k,\quad k\cdot j=-i,\quad i\cdot k=-j
\]
- **Perceptual Quaternion Model**:
\[
S(x,y)=\alpha C(x,y)+\beta T_{f}(x,y)i+\gamma G_{h}(x,y)j+\lambda G_{v}(x,y)k
\]
where:
- \(\alpha\) and \(\beta\) are scaling factors for adjusting the chrominance value \(C(x,y)\) and the residual energy.
- \(\gamma\) and \(\lambda\) are weighting factors for horizontal and vertical edges.
- \(T_{f}(x,y)\) is the residual energy of pixel \(P(x,y)\) in frame \(f\) relative to frame \(f - t\):
\[
T_{f}(x,y)=Y_{f}(x,y)+C_{f}(x,y)-Y_{f - t}(x,y)-C_{f - t}(x,y)
\]
- \(G_{h}(x,y)\) and \(G_{v}(x,y)\) are the horizontal and vertical gradients of pixel \(P(x,y)\) respectively.
- **Edge Information**:
- Edge energy:
\[
E(x,y)=\sqrt{G_{h}(x,y)^{2}+G_{v}(x,y)^{2}}
\]
- Direction:
\[
\text{Angle}(x,y)=\frac{180}{\pi}\arctan\left(\frac{G_{h}(x,y)}{G_{v}(x,y)}\right)
\]
### Conclusion
This paper proposes a perceptual video quality assessment method based on quaternions and a top - down framework by introducing chrominance information and spatio - temporal information. This method assesses video quality at multiple levels and verifies its superiority in practical applications through experiments.