Boosting Object Detection with Zero-Shot Day-Night Domain Adaptation

Zhipeng Du,Miaojing Shi,Jiankang Deng
2024-03-28
Abstract:Detecting objects in low-light scenarios presents a persistent challenge, as detectors trained on well-lit data exhibit significant performance degradation on low-light data due to low visibility. Previous methods mitigate this issue by exploring image enhancement or object detection techniques with real low-light image datasets. However, the progress is impeded by the inherent difficulties about collecting and annotating low-light images. To address this challenge, we propose to boost low-light object detection with zero-shot day-night domain adaptation, which aims to generalize a detector from well-lit scenarios to low-light ones without requiring real low-light data. Revisiting Retinex theory in the low-level vision, we first design a reflectance representation learning module to learn Retinex-based illumination invariance in images with a carefully designed illumination invariance reinforcement strategy. Next, an interchange-redecomposition-coherence procedure is introduced to improve over the vanilla Retinex image decomposition process by performing two sequential image decompositions and introducing a redecomposition cohering loss. Extensive experiments on ExDark, DARK FACE, and CODaN datasets show strong low-light generalizability of our method. Our code is available at <a class="link-external link-https" href="https://github.com/ZPDu/DAI-Net" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
This paper attempts to solve the problem of a significant decline in object detection performance in low - light environments. Specifically, due to issues such as low image visibility, color distortion, and increased noise caused by low - light environments, traditional object detectors perform poorly when processing low - light images. To overcome this challenge, the authors propose a zero - shot day - night domain adaptation method (Zero - Shot Day - Night Domain Adaptation, ZSDA) that does not require real low - light data, aiming to improve the performance of detectors in low - light conditions through generalization from bright scenes to low - light scenes. ### Main Contributions 1. **Reflectance Representation Learning Module**: The authors introduce an additional decoder module to enhance the invariance to illumination changes in existing object detection frameworks. This module strengthens the learning process through a specifically designed illumination invariance reinforcement strategy. 2. **Swap - Re - decomposition - Consistency Procedure**: An improved Retinex image decomposition process is proposed. Through two sequential image decompositions and the introduction of a re - decomposition consistency loss, the stability and accuracy of reflectance representation learning are improved. 3. **Extensive Experimental Verification**: Extensive experiments were carried out on multiple datasets, including ExDark (general object detection), DARK FACE (face detection), and CODaN (image classification). The experimental results show that this method significantly outperforms existing methods in various settings. ### Method Overview - **Zero - shot Day - Night Domain Adaptation (ZSDA)**: During the training phase, only bright images are used, and synthetic low - light images are generated through a physically inspired low - light degradation synthesis pipeline. The testing phase is evaluated on unseen real low - light images. - **DAI - Net Framework**: Combines a pre - trained Retinex decomposition network and an enhanced object detector. The reflectance representation learning module learns illumination - invariant information from bright and synthetic low - light images. - **Loss Function**: Includes detection loss, feature alignment loss, re - decomposition consistency loss, reflectance learning loss, and image decomposition loss, which jointly optimize the model. ### Experimental Results - **Face Detection**: The experimental results on the DARK FACE dataset show that DAI - Net significantly outperforms other methods in both zero - shot adaptation and fully - supervised settings. - **Object Detection**: Experiments on the ExDark dataset indicate that DAI - Net performs well on all object categories, especially on some difficult categories such as chairs, with a 7.6% improvement in mAP. - **Image Classification**: Experiments on the CODaN dataset further verify the generalization ability of the method, achieving a Top - 1 accuracy of 68.44%, which is significantly better than other methods. Through these contributions, this paper provides an effective method to solve the challenges of object detection in low - light environments and reduces the need for real low - light data.