Abstract:Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

Reconstructing a High Dynamic Range Image with a Deeply Unsupervised Fusion Model

Multi-exposure image fusion based on well-exposedness assessment

Dual-Focal Camera Hdr Imaging Based On Convolutional Neural Network

Ghost Detection And Removal Based On Super-Pixel Grouping In Exposure Fusion

Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes

Automatic Intermediate Generation With Deep Reinforcement Learning for Robust Two-Exposure Image Fusion

An Ultra-High-definition Multi-Exposure Image Fusion Method Based on Multi-Scale Feature Extraction

Progressive and Selective Fusion Network for High Dynamic Range Imaging

Learning Regularized Multi-Scale Feature Flow for High Dynamic Range Imaging

Efficient High Dynamic Range Video Using Multi-Exposure Cnn Flow

Deep SR-HDR: Joint Learning of Super-Resolution and High Dynamic Range Imaging for Dynamic Scenes

Ghost-Free Deep High-Dynamic-Range Imaging Using Focus Pixels for Complex Motion Scenes

DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs

Deep Coupled Feedback Network for Joint Exposure Fusion and Image Super-Resolution

Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image Restoration

Deep progressive feature aggregation network for multi-frame high dynamic range imaging

Deep HDR Deghosting by Motion-Attention Fusion Network

Coarse-to-fine multi-scale attention-guided network for multi-exposure image fusion

Deep Exposure Fusion with Deghosting via Homography Estimation and Attention Learning

Multi-exposure image fusion via deep perceptual enhancement

Multi-Exposure Image Fusion Algorithm Based on Improved Weight Function