Abstract:Existing deep learning techniques for image fusion either learn image mapping (LIM) directly, which renders them ineffective at preserving details due to the equal consideration to each pixel, or learn detail mapping (LDM), which only attains a limited level of performance because only details are used for reasoning. The recent lossless invertible network (INN) has demonstrated its detail-preserving ability. However, the direct applicability of INN to the image fusion task is limited by the volume-preserving constraint. Additionally, there is the lack of a consistent detail-preserving image fusion framework to produce satisfactory outcomes. To this aim, we propose a general paradigm for image fusion based on a novel conditional INN (named DCINN). The DCINN paradigm has three core components: a decomposing module that converts image mapping to detail mapping; an auxiliary network (ANet) that extracts auxiliary features directly from source images; and a conditional INN (CINN) that learns the detail mapping based on auxiliary features. The novel design benefits from the advantages of INN, LIM, and LDM approaches while avoiding their disadvantages. Particularly, using INN to LDM can easily meet the volume-preserving constraint while still preserving details. Moreover, since auxiliary features serve as conditional features, the ANet allows for the use of more than just details for reasoning without compromising detail mapping. Extensive experiments on three benchmark fusion problems, i.e., pansharpening, hyperspectral and multispectral image fusion, and infrared and visible image fusion, demonstrate the superiority of our approach compared with recent state-of-the-art methods. The code is available at https://github.com/wwhappylife/DCINN

Fusion-Cinn: A Generative Fusion Method for Multimodal Medical Images Based on Conditional Invertible Neural Network

FusionINN: Decomposable Image Fusion for Brain Tumor Monitoring

A General Paradigm with Detail-Preserving Conditional Invertible Network for Image Fusion

IGNFusion: An Unsupervised Information Gate Network for Multimodal Medical Image Fusion

Multimodal medical image fusion based on interval gradients and convolutional neural networks

A Medical Image Fusion Method Based on Convolutional Neural Networks

Medical Image Fusion Based on Adaptive Pulse Coupled Neural Networks and Nonsubsampled Contourlet Transform

Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

Medical image fusion with deep neural networks

CIRF: Coupled Image Reconstruction and Fusion Strategy for Deep Learning Based Multi-Modal Image Fusion

MCFNet: Multi-Layer Concatenation Fusion Network for Medical Images Fusion

A Medical Image Fusion Method Based on SIFT and Deep Convolutional Neural Network in the SIST Domain

Medical Image Fusion Via PCNN Based on Edge Preservation and Improved Sparse Representation in NSST Domain

A Generative Adversarial Network For Medical Image Fusion

C2IENet: Multi-branch medical image fusion based on contrastive constraint features and information exchange

Edge-Enhanced Dilated Residual Attention Network for Multimodal Medical Image Fusion

Multi-Source Information Exchange Encoding with PCNN for Medical Image Fusion

Feature-Motivated Simplified Adaptive PCNN-Based Medical Image Fusion Algorithm in NSST Domain

DDIFN: A Dual-discriminator Multi-modal Medical Image Fusion Network

MSAIF-Net: A Multistage Spatial Attention-Based Invertible Fusion Network for MR Images.

Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network