Abstract:Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

Asymmetric Stereo High Dynamic Range Imaging with Smartphone Cameras

A novel high dynamic range microscopic video system based on REC.2020

Dual-Focal Camera Hdr Imaging Based On Convolutional Neural Network

Robust Dual Images Super-Resolution.

Joint HDR Denoising and Fusion: A Real-World Mobile HDR Image Dataset

Automatic exposure selection and fusion for high-dynamic-range photography via smartphones

Asymmetric Stereo Color Transfer

DSR: Direct Self-rectification for Uncalibrated Dual-lens Cameras

Deep Hybrid Camera Deblurring for Smartphone Cameras

Handheld Multi-Frame Super-Resolution

High Dynamic Range and Super-Resolution from Raw Image Bursts

Disparity-based HDR imaging

Burst photography for high dynamic range and low-light imaging on mobile cameras

Efficient Hybrid Zoom using Camera Fusion on Mobile Phones

Pixel-wise Exposure Control for Single-Shot HDR Imaging: A Joint Optimization Approach

A Dual Camera System for High Spatiotemporal Resolution Video Acquisition

FloatingFusion: Depth from ToF and Image-stabilized Stereo Cameras

HDR Imaging with Spatially Varying Signal-to-Noise Ratios

Dual-Camera Joint Deblurring-Denoising

Ghost-Free Deep High-Dynamic-Range Imaging Using Focus Pixels for Complex Motion Scenes

Robust Depth Estimation from Auto Bracketed Images