Edge-enhanced Ultra High Definition Video Quality Assessment
Teng Jianxin,He Jiefeng,Yuan Jinchun,Xing Fengchuang,Wang Hanpin
DOI: https://doi.org/10.11834/jig.220648
2023-01-01
Journal of Image and Graphics
Abstract:ObjectiveThe 4 K（3 840 × 2 160 pixels） ultra high definition（UHD） video has been developing intensively in terms of emerging network and television technology. However, in respect of acquisition, compression, transmission and storage, the distortion-acquired issue is challenged due to the huge amount of UHD video data, rich edge and texture information, and high resolution. Our research is focused on an edge-enhanced UHD-VQA method because UHD-based video quality assessment（VQA） has become a crucial research domain in television broadcasting.MethodFirst, the input video frame is splitted to obtain 3 kinds of channels: 1) R, 2) G, and 3) B. Then, the edge detection operator is used to detect the edge information for each channel. The edge information of R, G and B channels is coordinated and the edge map of the video frame is obtained. To extract the spatial information of the video, human visual system（HVS） is targeted to develop its content-oriented. To extract the spatial information of the video frame further, each frame is input into the Image Net-1Ktrained Res Net-50. To reduce the dimension of features, a global pooling-derived feature maps are concatenated on 3aspects as mentioned below: 1) the feature maps is extracted and processed via recurrent unit-gated, 2) the min pooling and softmin pooling are used to process the features output, and 3) it is obtained and the prediction score can be calculated in terms of a sum of the weighted value. To extract multiple features, the masking-edged, content-oriented, and memorytemporal network modules are designed. Finally, to obtain the quality features and its video quality score-calculated, the features are melted into the fully connected layer network for dimensionality reduction. Due to the high definition and rich of edge details of UHD video, it is more likely to cause severe distortion at the edge. So, our edge-enhanced method can be adapted to the quality assessment of UHD video specially. At the same time, due to the introduction of content-oriented and time-domain hysteresis features, our method has its potentials for the quality assessment of more outdoor-relevant videos.ResultExperiments are compared to 5 popular methods on 4 datasets. We optimize some values on the 4 aspects: 1) 3. 9%SROCC（Spearman rank-order correlation coefficient） improvement and 3. 9%PLCC（Pearson’s linear correlation coefficient） improvement on Ko NVi D-1K, 2) 4. 2%SROCC improvement and 2. 2%PLCC improvement on DVL2021, 3)10. 0%SROCC improvement and 10. 1%PLCC improvement on LIVE-Qualcomm, and 4) 0. 6%SROCC improvement and0. 1%PLCC improvement on LSVQ. To demonstrate its generalization ability, a cross-dataset experiment is carried out as well. Furthermore, to optimize the effectiveness of edge information, we conduct an ablation study as well. Our illustrated network can be actually trained well to match the feature of edge masking without edge masking.ConclusionTo alleviate the edge-distorted, an edge-enhanced method is demonstrated to assess the quality of UHD video. At the same time, the content-oriented and time-domain hysteresis features are introduced to resolve the coordinated UHD-VQA problem. To detect edge information of video frames, the Canny operator is used and its configuration is sorted out. The training parameters are used to deal with the Heterogeneity problem in multiple video datasets. To verify the effectiveness of the proposed method, a large number of experiments are tested and compared to 4 popular video quality evaluation datasets（ UHD included）. The performance can be improved and reached to 10. 0%, and the smallest performance is gained 0. 1%as well. These experimental results show that edge information can optimize the performance of VQA methods greatly. The computational cost is optimized greatly since the optical flow method is not used. The future research direction can be predicted and concerned about more potential HVS features for the NR-VQA problem.