Abstract:Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

Research on Ghost-Free High Dynamic Range Image Fusion Technology of Space Non-cooperative Target

Ghost Detection And Removal Based On Super-Pixel Grouping In Exposure Fusion

Motion Blur Suppression Method Based on Dual Mode Imaging for Remote Sensing Image

Detail Preserving Exposure Fusion for A Dual Sensor Camera

Robust ghost-free multiexposure fusion for dynamic scenes.

Image tremble compensation for space camera

Fusion Algorithm of Laser-Point Clouds and Optical Images

Adaptive Switching Spatial-Temporal Fusion Detection for Remote Flying Drones

Optical-and-Radar Image Fusion for Dynamic Estimation of Spin Satellites

The Realistic Fusion of Multi-spectral Images

Pose Estimation of Non-Cooperative Space Targets Based on Cross-Source Point Cloud Fusion

High Precision Pose Estimation for Uncooperative Targets Based on Monocular Vision and 1D Laser Fusion

Visual Salience And Stack Extension Based Ghost Removal For High-Dynamic-Range Imaging

Accurate Pose Tracking for Uncooperative Targets via Data Fusion of Laser Scanner and Optical Camera

Robust Ghost-Free High-Dynamic-Range Imaging by Visual Salience Based Bilateral Motion Detection and Stack Extension Based Exposure Fusion.

A ghost-free multi-exposure image fusion using adaptive alignment for static and dynamic images

Deep Exposure Fusion with Deghosting via Homography Estimation and Attention Learning

Image Improvement Based on Sub-Pixel Post-Integration for A Staring Imaging System in Geostationary Orbit

Ghost-Free Deep High-Dynamic-Range Imaging Using Focus Pixels for Complex Motion Scenes

An Image Fusion Algorithm Using Spatial Consistency and Bands Correlation

Research on Spatial Target Recognition Method Based on Multi-source Sensor Fusion