Abstract:Pansharpening is a very debated spatio-spectral fusion problem. It refers to the fusion of a high spatial resolution panchromatic image with a lower spatial but higher spectral resolution multispectral image in order to obtain an image with high resolution in both the domains. In this article, we propose a novel variational optimization-based (VO) approach to address this issue incorporating the outcome of a deep convolutional neural network (DCNN). This solution can take advantages of both the paradigms. On one hand, higher performance can be expected introducing machine learning (ML) methods based on the training by examples philosophy into VO approaches. On other hand, the combination of VO techniques with DCNNs can aid the generalization ability of these latter. In particular, we formulate a $\ell _2$ -based proximal deep injection term to evaluate the distance between the DCNN outcome, and the desired high spatial resolution multispectral image. This represents the regularization term for our VO model. Furthermore, a new data fitting term measuring the spatial fidelity is proposed. Finally, the proposed convex VO problem is efficiently solved by exploiting the framework of the alternating direction method of multipliers (ADMM), thus guaranteeing the convergence of the algorithm. Extensive experiments both on simulated, and real datasets demonstrate that the proposed approach can outperform state-of-the-art spatio-spectral fusion methods, even showing a significant generalization ability. Please find the project page at https://liangjiandeng.github.io/Projects_Res/DMPIF_2020jstars.html .

Variational Osmosis for Non-linear Image Fusion

Variational Approach for Multi-Source Image Fusion

Infrared and Visible Image Fusion Via Variational Bayesian Approximation Method

Variational Image Matting with Local and Nonlocal Operators

A Variational Method for Multisource Remote-Sensing Image Fusion

A variational approach for multi-focus image fusion and visualization enhancement

A fractional osmosis model for image fusion

A color image fusion model by saturation-value total variation

Fusing Infrared and Visible Images Via a First-Order Model

Variational Bimodal Image Fusion with Data-Driven Tight Frame.

Variational Methods for Multimodal Image Matching

A Combined First and Second Order Variational Approach for Image Reconstruction

Image Blending with Osmosis

Variational Image Fusion Approach Based on TGV and Local Information

An Efficient Fusion Algorithm Combining Feature Extraction and Variational Optimization for CT and MR Images

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Adaptive Regularized Scheme for Remote Sensing Image Fusion

Variational Model for Image Segmentation.

A Variational Approach for Joint Image Recovery and Feature Extraction Based on Spatially Varying Generalised Gaussian Models

Perceptual Contrast-Based Image Fusion： A Variational Approach

An Iterative Approach for Image Fusion with Dynamic Gradient Sparsity and Anisotropic Spectral–spatial Total Variation