Abstract:As is well known, activity level measurement and fusion rule are two crucial factors in image fusion. For most existing fusion methods, either in spatial domain or in a transform domain like wavelet, the activity level measurement is essentially implemented by designing local filters to extract high-frequency details, and the calculated clarity information of different source images are then compared using some elaborately designed rules to obtain a clarity/focus map. Consequently, the focus map contains the integrated clarity information, which is of great significance to various image fusion issues, such as multi-focus image fusion, multi-modal image fusion, etc. However, in order to achieve a satisfactory fusion performance, these two tasks are usually difficult to finish. In this study, we address this problem with a deep learning approach, aiming to learn a direct mapping between source images and focus map. To this end, a deep convolutional neural network (CNN) trained by high-quality image patches and their blurred versions is adopted to encode the mapping. The main novelty of this idea is that the activity level measurement and fusion rule can be jointly generated through learning a CNN model, which overcomes the difficulty faced by the existing fusion methods. Based on the above idea, a new multi-focus image fusion method is primarily proposed in this paper. Experimental results demonstrate that the proposed method can obtain state-of-the-art fusion performance in terms of both visual quality and objective assessment. The computational speed of the proposed method using parallel computing is fast enough for practical usage. The potential of the learned CNN model for some other-type image fusion issues is also briefly exhibited in the experiments.

The Multi-Focus-Image-Fusion Method Based on Convolutional Neural Network and Sparse Representation.

The multi-focus image fusion method based on CNN and SR

CCSR-Net: Unfolding Coupled Convolutional Sparse Representation for Multi-focus Image Fusion.

Multi-focus image fusion based on non-negative sparse representation and patch-level consistency rectification

Multi-focus image fusion with a deep convolutional neural network

Multi-focus Image Fusion with Convolutional Neural Network Based on Dempster-Shafer Theory

Multi-Focus Color Image Fusion Algorithm Based on Super-Resolution Reconstruction and Focused Area Detection

Multi-focus Image Fusion Based on Fully Convolutional Networks.

Multi-Focus Image Fusion with a Natural Enhancement Via a Joint Multi-Level Deeply Supervised Convolutional Neural Network

Multi-focus image fusion based on multi-scale sparse representation

A Novel Sparse Representation Based Fusion Approach for Multi-Focus Images

A Novel Sparse-Representation-based Multi-Focus Image Fusion Approach

A Multi-Focus Image Fusion Framework Based on Multi-Scale Sparse Representation in Gradient Domain

Multi-focus Image Fusion Using Deep Support Value Convolutional Neural Network

Multi-Scale Visual Attention Deep Convolutional Neural Network for Multi-Focus Image Fusion

Convolutional Neural Network Based Multi-Focus Image Fusion

A Novel Multi-Focus Image Fusion Method Based on Stochastic Coordinate Coding and Local Density Peaks Clustering

Multi-focus Image Fusion Based on Robust Principal Component Analysis and Pulse-Coupled Neural Network

Multi-focus Image Fusion with Sparse Feature Based Pulse Coupled Neural Network

Multi-Focus Image Fusion Based on Adaptive Dual-Channel Spiking Cortical Model in Non-Subsampled Shearlet Domain.

Multi-focus Image Fusion Combining Focus-Region-level Partition and Pulse-Coupled Neural Network