Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving

Jinlong Li,Baolu Li,Zhengzhong Tu,Xinyu Liu,Qing Guo,Felix Juefei-Xu,Runsheng Xu,Hongkai Yu

2024-04-07

Abstract:Vision-centric perception systems for autonomous driving have gained considerable attention recently due to their cost-effectiveness and scalability, especially compared to LiDAR-based systems. However, these systems often struggle in low-light conditions, potentially compromising their performance and safety. To address this, our paper introduces LightDiff, a domain-tailored framework designed to enhance the low-light image quality for autonomous driving applications. Specifically, we employ a multi-condition controlled diffusion model. LightDiff works without any human-collected paired data, leveraging a dynamic data degradation process instead. It incorporates a novel multi-condition adapter that adaptively controls the input weights from different modalities, including depth maps, RGB images, and text captions, to effectively illuminate dark scenes while maintaining context consistency. Furthermore, to align the enhanced images with the detection model's knowledge, LightDiff employs perception-specific scores as rewards to guide the diffusion training process through reinforcement learning. Extensive experiments on the nuScenes datasets demonstrate that LightDiff can significantly improve the performance of several state-of-the-art 3D detectors in night-time conditions while achieving high visual quality scores, highlighting its potential to safeguard autonomous driving.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of image enhancement under low-light conditions in autonomous driving. Specifically:

- **Research Background**: Vision-based autonomous driving perception systems have garnered attention due to their cost-effectiveness and scalability. However, these systems often perform poorly under low-light conditions, potentially jeopardizing driving safety.
- **Main Objective**: A new framework named LightDiff is proposed, aiming to enhance image quality in low-light environments without relying on manually collected paired data, thereby improving the safety and performance of autonomous vehicles during nighttime driving.
- **Technical Approach**: A multi-condition controlled diffusion model is employed, utilizing a dynamic data degradation process to generate synthetic day-night image pairs. This approach combines various modalities such as depth maps, RGB images, and text captions to maintain scene context consistency.
- **Improvements**: The diffusion training process is guided by reinforcement learning, using perception model-specific scores as rewards to ensure that the enhanced images are consistent with the knowledge of the detection model.

Extensive experiments on the nuScenes dataset validate that LightDiff can significantly improve the performance of multiple state-of-the-art 3D detectors at night while achieving high visual quality scores, demonstrating its potential in ensuring autonomous driving safety.