Abstract:High dynamic range imaging (HDRI) is an essential task in remote sensing, enhancing low dynamic range (LDR) remote sensing images and benefiting downstream tasks, such as object detection and image segmentation. However, conventional frame-based HDRI methods may encounter challenges in real-world scenarios due to the limited information inherent in a single image captured by conventional cameras. In this paper, an event-based remote sensing HDR imaging framework is proposed to address this problem, denoted as ERS-HDRI, which reconstructs the remote sensing HDR image from a single-exposure LDR image and its concurrent event streams. The proposed ERS-HDRI leverages a coarse-to-fine framework, incorporating the event-based dynamic range enhancement (E-DRE) network and the gradient-enhanced HDR reconstruction (G-HDRR) network. Specifically, to efficiently achieve dynamic range fusion from different domains, the E-DRE network is designed to extract the dynamic range features from LDR frames and events and perform intra- and cross-attention operations to adaptively fuse multi-modal data. A denoise network and a dense feature fusion network are then employed for the generation of the coarse, clean HDR image. Then, the G-HDRR network, with its gradient enhancement module and multiscale fusion module, performs structure enforcement on the coarse HDR image and generates a fine informative HDR image. In addition, this work introduces a specialized hybrid imaging system and a novel, real-world event-based remote sensing HDRI dataset that contains aligned remote sensing LDR images, remote sensing HDR images, and concurrent event streams for evaluation. Comprehensive experiments have demonstrated the effectiveness of the proposed method. Specifically, it improves state-of-the-art PSNR by about 30% and the SSIM score by about 9% on the real-world dataset.

What problem does this paper attempt to address?

The paper primarily aims to address the issues encountered in High Dynamic Range Imaging (HDR) in remote sensing, particularly how to utilize data from event cameras to improve the quality of traditional Low Dynamic Range (LDR) remote sensing images. ### Core Issues Addressed by the Paper 1. **Domain Gap**: There is a significant difference between traditional RGB cameras and event cameras, which makes it challenging to effectively fuse data obtained from these two different types of sensors. The paper proposes a method to narrow this domain gap, achieving adaptive fusion between optical images and event streams. 2. **Light Attenuation**: In remote sensing images captured from high altitudes, the intensity of light decreases with distance, posing a challenge for event cameras in capturing detailed information. Therefore, reconstructing detailed structures in underexposed or overexposed areas becomes difficult. 3. **High-Quality HDR Image Reconstruction under Single Exposure Conditions**: Traditional multi-exposure HDR methods may produce ghosting effects when dealing with moving objects; while single-exposure HDR methods can avoid ghosting, they still face challenges in reconstructing HDR images due to limited information. ### Main Contributions - Proposes an event-based remote sensing HDR imaging framework (ERS-HDRI) that can reconstruct high-quality HDR images from single-exposure LDR images and their concurrent event streams. - Adopts a coarse-to-fine strategy to effectively achieve dynamic range enhancement and structure enhancement, addressing both the domain gap issue and mitigating the effects of light attenuation. - Constructs a hybrid imaging system and a new remote sensing event-based HDR imaging dataset, which includes aligned LDR images, HDR images, and concurrent event streams, for evaluating the proposed method. ### Method Overview - **Event-based Dynamic Range Enhancement Network (E-DRE)**: Aims to recover the basic information of HDR images by fusing LDR image features and event stream features. This network includes steps such as feature extraction, denoising, and dense feature fusion, and introduces a self-attention mechanism to improve the feature fusion process. - **Gradient-enhanced HDR Reconstruction Network (G-HDRR)**: Further enhances image structure information based on the coarse reconstruction. This network uses the gradients of the coarse HDR image and short-time event frames to generate a structure attention map, thereby enhancing details in low-contrast areas. ### Optimization Strategy - Joint optimization is performed using pixel-level and feature-level loss functions (such as L1 loss and perceptual loss) as well as Generative Adversarial Network (GAN) loss to ensure the quality and color accuracy of the reconstructed images. In summary, the paper effectively addresses a series of challenges in remote sensing imaging by proposing a new event-based HDR imaging method, providing strong technical support for improving the quality and application of remote sensing images.

ERS-HDRI: Event-Based Remote Sensing HDR Imaging

HDR Imaging for Dynamic Scenes with Events

Event-assisted 12-stop HDR Imaging of Dynamic Scene

Single-Image HDR Reconstruction Assisted Ghost Suppression and Detail Preservation Network for Multi-Exposure HDR Imaging

Color Event Enhanced Single-Exposure HDR Imaging

HDRT: Infrared Capture for HDR Imaging

NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results

SJ-HD^2R: Selective Joint High Dynamic Range and Denoising Imaging for Dynamic Scenes

Learning Event Guided High Dynamic Range Video Reconstruction

EventHDR: from Event to High-Speed HDR Videos and Beyond

Efficient HDR Reconstruction from Real-World Raw Images

RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image

EiffHDR: An Efficient Network for Multi-Exposure High Dynamic Range Imaging.

Event-based Asynchronous HDR Imaging by Temporal Incident Light Modulation

HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains

Deep Joint Demosaicing and High Dynamic Range Imaging Within a Single Shot

Deep HDR Imaging via A Non-local Network

HDR Light Field Imaging of Dynamic Scenes: A Learning-based Method and A Benchmark Dataset