FastLLVE: Real-Time Low-Light Video Enhancement with Intensity-Aware Lookup Table

Wenhao Li,Guangyang Wu,Wenyi Wang,Peiran Ren,Xiaohong Liu

DOI: https://doi.org/10.1145/3581783.3611933

2023-08-13

Abstract:Low-Light Video Enhancement (LLVE) has received considerable attention in recent years. One of the critical requirements of LLVE is inter-frame brightness consistency, which is essential for maintaining the temporal coherence of the enhanced video. However, most existing single-image-based methods fail to address this issue, resulting in flickering effect that degrades the overall quality after enhancement. Moreover, 3D Convolution Neural Network (CNN)-based methods, which are designed for video to maintain inter-frame consistency, are computationally expensive, making them impractical for real-time applications. To address these issues, we propose an efficient pipeline named FastLLVE that leverages the Look-Up-Table (LUT) technique to maintain inter-frame brightness consistency effectively. Specifically, we design a learnable Intensity-Aware LUT (IA-LUT) module for adaptive enhancement, which addresses the low-dynamic problem in low-light scenarios. This enables FastLLVE to perform low-latency and low-complexity enhancement operations while maintaining high-quality results. Experimental results on benchmark datasets demonstrate that our method achieves the State-Of-The-Art (SOTA) performance in terms of both image quality and inter-frame brightness consistency. More importantly, our FastLLVE can process 1,080p videos at $\mathit{50+}$ Frames Per Second (FPS), which is $\mathit{2 \times}$ faster than SOTA CNN-based methods in inference time, making it a promising solution for real-time applications. The code is available at <a class="link-external link-https" href="https://github.com/Wenhao-Li-777/FastLLVE" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The paper attempts to address several key challenges in Low-Light Video Enhancement (LLVE), specifically including:

1. **Ill-posed Problem**: In low-light videos, due to the lower dynamic range of the color space, similar color inputs may correspond to different target colors, leading to a one-to-one mapping problem, which is difficult to solve in complex scenes.
2. **Brightness Consistency**: Maintaining brightness consistency in the output video is crucial for achieving high perceptual quality. However, existing alignment methods often fail to accurately align adjacent frames, resulting in unstable output; and self-consistency loss functions cannot solve the fundamental instability problem.
3. **Efficiency**: Although 3D Convolutional Neural Network (3D-CNN) methods perform well in video enhancement tasks, their high computational complexity makes them difficult to apply to real-time processing.

To address these issues, the authors propose a new framework called FastLLVE, which achieves real-time low-light video enhancement by using Intensity-Aware Lookup Table (IA-LUT) technology. Specifically, the IA-LUT module can handle the low dynamic range problem and naturally maintain inter-frame brightness consistency through pixel-level lookup table transformation. Additionally, this method is computationally efficient and suitable for real-time video enhancement. Experimental results show that FastLLVE achieves state-of-the-art performance on benchmark datasets and reaches real-time processing speeds of over 50 frames per second in 1080p video processing.