Abstract:Artificial lights commonly leave strong lens flare artifacts on the images captured at night, degrading both the visual quality and performance of vision algorithms. Existing flare removal approaches mainly focus on removing daytime flares and fail in nighttime cases. Nighttime flare removal is challenging due to the unique luminance and spectrum of artificial lights, as well as the diverse patterns and image degradation of the flares. The scarcity of the nighttime flare removal dataset constraints the research on this crucial task. In this paper, we introduce Flare7K++, the first comprehensive nighttime flare removal dataset, consisting of 962 real-captured flare images (Flare-R) and 7,000 synthetic flares (Flare7K). Compared to Flare7K, Flare7K++ is particularly effective in eliminating complicated degradation around the light source, which is intractable by using synthetic flares alone. Besides, the previous flare removal pipeline relies on the manual threshold and blur kernel settings to extract light sources, which may fail when the light sources are tiny or not overexposed. To address this issue, we additionally provide the annotations of light sources in Flare7K++ and propose a new end-to-end pipeline to preserve the light source while removing lens flares. Our dataset and pipeline offer a valuable foundation and benchmark for future investigations into nighttime flare removal studies. Extensive experiments demonstrate that Flare7K++ supplements the diversity of existing flare datasets and pushes the frontier of nighttime flare removal towards real-world scenarios.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in night - time photography, the strong lens flare phenomenon caused by artificial light sources seriously affects the visual quality of images and the performance of computer vision algorithms. Specifically, night - time flare removal faces the following challenges: 1. **Diverse flare patterns**: The patterns of night - time flare become very complex due to factors such as the position of light sources, different spectra, lens manufacturing defects, and scratches and oil stains in daily use. 2. **Light scattering and interference**: The scattering and interference of light of different wavelengths in small optical structures can lead to rainbow - like halos and color moiré patterns, which further increase the difficulty of flare modeling. 3. **Limitations of existing methods**: Existing flare removal methods mainly focus on daytime flare or specific types of flare and are unable to handle the complex and diverse flare patterns at night. To address these challenges, the authors constructed a large - scale night - time flare dataset **Flare7K++** and proposed a new end - to - end framework for retaining light sources while removing lens flare. The specific contributions are as follows: 1. **Constructing the Flare7K++ dataset**: This dataset contains 962 real - shot flare images (Flare - R) and 7,000 synthetic flare images (Flare7K), which can better simulate real - world night - time flare. 2. **Providing light source annotations**: Light source annotations are provided for all images, and a network is designed to extract light sources from real - shot flare images. 3. **New end - to - end framework**: Based on the provided light source annotations, a new end - to - end framework is designed that can retain light sources while removing lens flare. 4. **Introducing new evaluation metrics**: Masks of different flare components in test images are manually labeled, and two new evaluation metrics, G - PSNR and S - PSNR, are introduced to reflect the restoration effect in flare and streak - damaged areas. 5. **Downstream task evaluation**: The effectiveness of the method is evaluated on tasks such as stereo matching, optical flow estimation, and semantic segmentation related to night - time autonomous driving. Through these contributions, this paper provides a valuable benchmark and foundation for night - time flare removal research.

Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond

Flare7K: A Phenomenological Nighttime Flare Removal Dataset

A Data Generation Method for Image Flare Removal Based on Similarity and Centrosymmetric Effect

Harmonizing Light and Darkness: A Symphony of Prior-guided Data Synthesis and Adaptive Focus for Nighttime Flare Removal

Nighttime Image Dehazing by Render.

Toward Real Flare Removal: A Comprehensive Pipeline and A New Benchmark

Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior

Improving Lens Flare Removal with General Purpose Pipeline and Multiple Light Sources Recovery

Toward Blind Flare Removal Using Knowledge-Driven Flare-Level Estimator

MFDNet: Multi-Frequency Deflare Network for Efficient Nighttime Flare Removal

Understanding and Tackling Scattering and Reflective Flare for Mobile Camera Systems

Flare Removal Model Based on Sparse-UFormer Networks

Ground-to-air Aircraft Infrared Image Deblurring Based on Imaging Degradation Simulation

Difflare: Removing Image Lens Flare with Latent Diffusion Model

GN-FR:Generalizable Neural Radiance Fields for Flare Removal

Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

MIPI 2023 Challenge on Nighttime Flare Removal: Methods and Results

CAM-FRN: Class Attention Map-Based Flare Removal Network in Frontal-Viewing Camera Images of Vehicles

Toward Flare-Free Images: A Survey

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal