Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancement

Yunlong Dong,Xiaohong Liu,Yixuan Gao,Xunchu Zhou,Tao Tan,Guangtao Zhai
2023-08-06
Abstract:Recently, Users Generated Content (UGC) videos becomes ubiquitous in our daily lives. However, due to the limitations of photographic equipments and techniques, UGC videos often contain various degradations, in which one of the most visually unfavorable effects is the underexposure. Therefore, corresponding video enhancement algorithms such as Low-Light Video Enhancement (LLVE) have been proposed to deal with the specific degradation. However, different from video enhancement algorithms, almost all existing Video Quality Assessment (VQA) models are built generally rather than specifically, which measure the quality of a video from a comprehensive perspective. To the best of our knowledge, there is no VQA model specially designed for videos enhanced by LLVE algorithms. To this end, we first construct a Low-Light Video Enhancement Quality Assessment (LLVE-QA) dataset in which 254 original low-light videos are collected and then enhanced by leveraging 8 LLVE algorithms to obtain 2,060 videos in total. Moreover, we propose a quality assessment model specialized in LLVE, named Light-VQA. More concretely, since the brightness and noise have the most impact on low-light enhanced VQA, we handcraft corresponding features and integrate them with deep-learning-based semantic features as the overall spatial information. As for temporal information, in addition to deep-learning-based motion features, we also investigate the handcrafted brightness consistency among video frames, and the overall temporal information is their concatenation. Subsequently, spatial and temporal information is fused to obtain the quality-aware representation of a video. Extensive experimental results show that our Light-VQA achieves the best performance against the current State-Of-The-Art (SOTA) on LLVE-QA and public dataset. Dataset and Codes can be found at <a class="link-external link-https" href="https://github.com/wenzhouyidu/Light-VQA" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition,Image and Video Processing
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that the existing video quality assessment (VQA) models are unable to conduct effective quality assessment specifically for low - light video enhancement (LLVE) algorithms. Specifically, although there are already many general - purpose VQA models, these models are not specifically designed to evaluate the video quality after low - light video enhancement processing. Therefore, the quality assessment of low - light video enhancement results still lacks specificity and accuracy. To fill this gap, the authors propose the following solutions: 1. **Construct a specialized dataset**: The authors constructed a dataset named LLVE - QA, which contains 254 original low - light videos and 2,060 videos enhanced by 8 different LLVE algorithms. Each video has a mean opinion score (MOS) obtained through subjective experiments as a benchmark for the true perceptual quality. 2. **Propose a specialized quality assessment model**: Based on the constructed dataset, the authors propose a new quality assessment model - Light - VQA. This model is specifically designed for low - light video enhancement and can combine hand - crafted features (such as brightness, noise) and deep - learning features (such as semantic features, motion features) to comprehensively evaluate video quality. Specifically: - **Spatial information extraction module**: Extract deep - learning features (using pre - trained Swin Transformer) and hand - crafted features (brightness and noise) from key frames. - **Temporal information extraction module**: Extract deep - learning features (using pre - trained SlowFast network) and hand - crafted features (brightness consistency) from video clips. - **Feature fusion module**: Fuse the spatial and temporal information together to form a quality - aware representation. - **Quality regression module**: Regress the fused features into video quality scores through a fully - connected layer. 3. **Verify the model performance**: Through extensive experimental verification, Light - VQA outperforms the existing 6 state - of - the - art VQA models on the LLVE - QA dataset and public datasets. In summary, this paper aims to solve the problem of inaccurate evaluation of low - light video enhancement results by existing VQA models, and proposes a new dedicated dataset and quality assessment model for this purpose.