Abstract:Capturing real world lighting is a long standing challenge in imaging and most practical methods acquire High Dynamic Range (HDR) images by either fusing multiple exposures, or boosting the dynamic range of Standard Dynamic Range (SDR) images. Multiple exposure capture is problematic as it requires longer capture times which can often lead to ghosting problems. The main alternative, inverse tone mapping is an ill-defined problem that is especially challenging as single captured exposures usually contain clipped and quantized values, and are therefore missing substantial amounts of content. To alleviate this, we propose a new approach, High Dynamic Range Thermal (HDRT), for HDR acquisition using a separate, commonly available, thermal infrared (IR) sensor. We propose a novel deep neural method (HDRTNet) which combines IR and SDR content to generate HDR images. HDRTNet learns to exploit IR features linked to the RGB image and the IR-specific parameters are subsequently used in a dual branch method that fuses features at shallow layers. This produces an HDR image that is significantly superior to that generated using naive fusion approaches. To validate our method, we have created the first HDR and thermal dataset, and performed extensive experiments comparing HDRTNet with the state-of-the-art. We show substantial quantitative and qualitative quality improvements on both over- and under-exposed images, showing that our approach is robust to capturing in multiple different lighting conditions.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the key challenges in high - dynamic - range (HDR) imaging, that is, how to generate high - quality HDR images without relying on multiple differently - exposed images. Specifically, the paper proposes a novel method to generate HDR images by combining infrared (IR) images and standard - dynamic - range (SDR) images, thus overcoming several major problems existing in the existing methods: 1. **Problems with multi - exposure fusion methods**: - **Long - exposure**: It requires a relatively long shooting time and is prone to the ghosting problem, especially in the presence of moving objects or people. - **Medium dynamic range**: Although it can capture more details, it still cannot reach the true high - dynamic range. 2. **Problems with inverse tone mapping (ITM)**: - **Information loss**: A single SDR image usually contains clipped and quantized values, resulting in a large amount of content loss, especially in over - exposed or under - exposed areas. - **Ill - posed problem**: Inverse tone mapping is an ill - posed problem, and it is difficult to recover enough information from a single image to generate a high - quality HDR image. To solve these problems, the author proposes a new method - **High Dynamic Range Thermal (HDRT)**, which combines infrared images obtained by thermal infrared sensors with visible - light images (RGB images) and generates HDR images through a deep - learning model (HDRTNet). The main advantages of this method are: - **Reducing the ghosting problem**: Since infrared images are not affected by excessive or insufficient visible light, they can provide stable information supplementation. - **Improving image quality**: By fusing the information in infrared and visible - light images, it is possible to better recover the details in over - exposed and under - exposed areas. - **Shortening the shooting time**: Only one shot is required to obtain a high - quality HDR image, avoiding the complexity and time consumption brought by multiple exposures. ### Specific contributions of the paper 1. **Introducing infrared - domain - enhanced HDR imaging for the first time**: Using the temperature information provided by infrared images to supplement the missing details in visible - light images. 2. **Proposing a new deep - learning method, HDRTNet**: This method fuses infrared and RGB images through a two - branch structure to generate high - quality HDR images. 3. **Creating the first aligned infrared and HDR dataset**: It contains 10,000 RGB images (including HDR and SDR) and 10,000 infrared images for further research on RGBT - fused HDR imaging. 4. **Demonstrating significant performance improvement**: Through quantitative and qualitative experiments, it is proved that HDRTNet is superior to existing methods in processing over - exposed and under - exposed images. In conclusion, this paper proposes an innovative method to generate high - quality HDR images by combining infrared and visible - light images, solves multiple problems existing in traditional methods, and provides new ideas and tools for future HDR imaging research.

HDRT: Infrared Capture for HDR Imaging

CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain

NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results

ERS-HDRI: Event-Based Remote Sensing HDR Imaging

Implementing evidence-based practice in Taiwanese nursing homes: attitudes and perceived barriers and facilitators.

HDR-NeRF: High Dynamic Range Neural Radiance Fields

RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image

HDR Light Field Imaging of Dynamic Scenes: A Learning-based Method and A Benchmark Dataset

High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe Saturation

HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains

HDR Imaging for Dynamic Scenes with Events

SJ-HD^2R: Selective Joint High Dynamic Range and Denoising Imaging for Dynamic Scenes

Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation

Deep SR-HDR: Joint Learning of Super-Resolution and High Dynamic Range Imaging for Dynamic Scenes

Efficient HDR Reconstruction from Real-World Raw Images

A New Journey from SDRTV to HDRTV

HDRTransDC: High Dynamic Range Image Reconstruction with Transformer Deformation Convolution

Deep high dynamic range imaging of dynamic scenes

Wavelet-Based Network For High Dynamic Range Imaging