Abstract:Event cameras with high dynamic range ensure scene capture even in low-light conditions. However, night events exhibit patterns different from those captured during the day. This difference causes performance degradation when applying night events to a model trained solely on day events. This limitation persists due to a lack of annotated night events. To overcome the limitation, we aim to alleviate data imbalance by translating annotated day data into night events. However, generating events from different modalities challenges reproducing their unique properties. Accordingly, we propose an unpaired event-to-event day-to-night translation model that effectively learns to map from one domain to another using Diffusion GAN. The proposed translation model analyzes events in spatio-temporal dimension with wavelet decomposition and disentangled convolution layers. We also propose a new temporal contrastive learning with a novel shuffling and sampling strategy to regularize temporal continuity. To validate the efficacy of the proposed methodology, we redesign metrics for evaluating events translated in an unpaired setting, aligning them with the event modality for the first time. Our framework shows the successful day-to-night event translation while preserving the characteristics of events. In addition, through our translation method, we facilitate event-based modes to learn about night events by translating annotated day events into night events. Our approach effectively mitigates the performance degradation of applying real night events to downstream tasks. The code is available at <a class="link-external link-https" href="https://github.com/jeongyh98/UDNET" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to alleviate the data imbalance problem through unpaired day - to - night event conversion, thereby improving the performance of event - based networks under night - time conditions**. Specifically, the event camera can capture scenes under low - light conditions, but the pattern of night - time events is different from that of day - time events. This leads to a performance degradation when a model trained only with day - time events is applied to night - time events. Due to the lack of labeled night - time event data, this performance degradation problem has persisted. To solve this problem, the authors propose an unpaired day - to - night event conversion model, aiming to convert the labeled day - time data into night - time events to alleviate the data imbalance problem and improve the performance of the model under night - time conditions. ### Key Challenges 1. **Data Imbalance**: Night - time event data is scarce, and it is difficult to obtain sufficient labeled data. 2. **Modal Difference**: Generating events needs to maintain their unique spatio - temporal characteristics, rather than simply generating events from images. 3. **Temporal Continuity**: Events have temporal continuity, and existing methods are difficult to handle this well. ### Solutions The authors propose an unpaired event - to - event day - to - night conversion model based on Diffusion GAN, which specifically includes the following key points: - **Wavelet Decomposition and Decoupled Convolutional Layers**: Through the analysis of events in the spatio - temporal dimension, wavelet decomposition and decoupled convolutional layers are used to effectively learn the mapping from day - to - night. - **Temporal Contrastive Learning**: A new temporal contrastive learning method is introduced, which regularizes temporal continuity by randomly shuffling the temporal order of events. - **Redesign of Evaluation Metrics**: In order to verify the effect of unpaired conversion, the authors redesign the evaluation metrics to make them more in line with the characteristics of the event modality. Through these methods, the authors successfully achieve the day - to - night event conversion, and verify the effectiveness of this method in multiple downstream tasks, significantly improving the performance of the model under night - time conditions. ### Summary The main contribution of this paper lies in proposing an effective unpaired day - to - night event conversion method, solving the performance degradation problem caused by the scarcity of night - time event data, and ensuring the temporal continuity and unique properties of the generated events through various technical means.

Towards Robust Event-based Networks for Nighttime via Unpaired Day-to-Night Event Translation

See Clearer at Night: Towards Robust Nighttime Semantic Segmentation Through Day-Night Image Conversion

Modular Degradation Simulation and Restoration for Under-Display Camera

Night-to-Day Image Translation for Retrieval-based Localization

Seeing Motion at Nighttime with an Event Camera

Enhanced Night-to-Day Image Conversion Using CycleGAN-Based Base-Detail Paired Training

Semantic and Geometric-Aware Day-to-Night Image Translation Network

Let There be Light: Improved Traffic Surveillance via Detail Preserving Night-to-Day Transfer

Disentangled Contrastive Image Translation for Nighttime Surveillance

Night-to-Day Translation via Illumination Degradation Disentanglement

GAN-Based Day-to-Night Image Style Transfer for Nighttime Vehicle Detection

Nighttime vehicle detection algorithm based on image translation technology1

Towards Real-world Event-guided Low-light Video Enhancement and Deblurring

Translating Multispectral Imagery to Nighttime Imagery via Conditional Generative Adversarial Networks

Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera

Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning

Unsupervised Domain Adaptation for Nighttime Aerial Tracking

Unpaired Image-to-Image Translation with Diffusion Adversarial Network

From Night to Day: GANs Based Low Quality Image Enhancement

Nighttime road scene image enhancement based on cycle-consistent generative adversarial network

Day-to-Night Image Synthesis for Training Nighttime Neural ISPs