Abstract:Convolutional dictionary learning has become increasingly popular in signal and image processing for its ability to overcome the limitations of traditional patch-based dictionary learning. Although most studies on convolutional dictionary learning mainly focus on the unimodal case, real-world image processing tasks usually involve images from multiple modalities, e.g., visible and near-infrared (NIR) images. Thus, it is necessary to explore convolutional dictionary learning across different modalities. In this paper, we propose a novel multi-modal convolutional dictionary learning algorithm, which efficiently correlates different image modalities and fully considers neighborhood information at the image level. In this model, each modality is represented by two convolutional dictionaries, in which one dictionary is for common feature representation and the other is for unique feature representation. The model is constrained by the requirement that the convolutional sparse representations (CSRs) for the common features should be the same across different modalities, considering that these images are captured from the same scene. We propose a new training method based on the alternating direction method of multipliers (ADMM) to alternatively learn the common and unique dictionaries in the discrete Fourier transform (DFT) domain. We show that our model converges in less than 20 iterations between the convolutional dictionary updating and the CSRs calculation. The effectiveness of the proposed dictionary learning algorithm is demonstrated on various multimodal image processing tasks, achieves better performance than both dictionary learning methods and deep learning based methods with limited training data.

Deep M2CDL: Deep Multi-Scale Multi-Modal Convolutional Dictionary Learning Network.

Multi-View Analysis Dictionary Learning for Image Classification.

Multi-modal deep convolutional dictionary learning for image denoising

Multi-Modal Convolutional Dictionary Learning.

Convolutional Dictionary Pair Learning Network for Image Representation Learning

A Novel Multi-Layer Discriminative Dictionary Learning Approach for Image Classification

Deep Dynamic Memory Augmented Attentional Dictionary Learning for Image Denoising

When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data

Multi-level Discriminative Dictionary Learning with Application to Large Scale Image Classification.

Intra- and Inter-Class Induced Discriminative Deep Dictionary Learning for Visual Recognition

MDFL: Multi-domain Diffusion-driven Feature Learning

Deep Micro-Dictionary Learning and Coding Network

Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction

Interpretable Structured Multi-modal Deep Neural Network

MADPL-net: Multi-layer Attention Dictionary Pair Learning Network for Image Classification

DCMFNet: Deep Cross-Modal Fusion Network for Different Modalities with Iterative Gated Fusion

Multi-layer Discriminative Dictionary Learning with Locality Constraint for Image Classification.

Deep Mamba Multi-modal Learning

Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion

Multi-view Low-Rank Dictionary Learning for Image Classification

A Dimensional Structure based Knowledge Distillation Method for Cross-Modal Learning