Abstract:Hyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes. We regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking. Quantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis. Our results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

Optimized High-Frequency Based Interpolation for Multispectral Demosaicking

Adaptive Multispectral Demosaicking Based on Frequency-Domain Analysis of Spectral Correlation

PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

An Efficient Adaptive Interpolation for Bayer CFA Demosaicking

Color demosaicking via fully directional estimation

An Improved Color Image Demosaicking Algorithm

Low Cost Edge Sensing for High Quality Demosaicking

Joint optimization of multispectral filter arrays and demosaicking for pathological images

Bayer Demosaicking With Polynomial Interpolation

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging

Optimized Color Filter Arrays for Sparse Representation-Based Demosaicking

Fast Multispectral Imaging by Spatial Pixel-Binning and Spectral Unmixing

Multispectral Demosaicing Based on Iterative-Linear-Regression Model for Estimating Pseudo-Panchromatic Image

An Efficient Transformer For Demosaicing Via Compressed Multi-Branch Attention Mechanism.

LMMSE frequency merging for demosaicking

Universal Demosaicking of Color Filter Arrays

Frequency Selection and Merging with Universal Matrices for Color Filter Array Demosaicking

A Hardware Friendly Demosaicking Algorithm Based on Edge Sensing

T-Hy-Demosaicing: Hyperspectral Reconstruction Via Tensor Subspace Representation Under Orthogonal Transformation