Abstract:Neural network architectures for image demosaicing have been become more and more complex. This results in long training periods of such deep networks and the size of the networks is huge. These two factors prevent practical implementation and usage of the networks in real-time platforms, which generally only have limited resources. This study investigates the effectiveness of neural network architectures in hyperspectral image demosaicing. We introduce a range of network models and modifications, and compare them with classical interpolation methods and existing reference network approaches. The aim is to identify robust and efficient performing network architectures. Our evaluation is conducted on two datasets, "SimpleData" and "SimRealData," representing different degrees of realism in multispectral filter array (MSFA) data. The results indicate that our networks outperform or match reference models in both datasets demonstrating exceptional performance. Notably, our approach focuses on achieving correct spectral reconstruction rather than just visual appeal, and this emphasis is supported by quantitative and qualitative assessments. Furthermore, our findings suggest that efficient demosaicing solutions, which require fewer parameters, are essential for practical applications. This research contributes valuable insights into hyperspectral imaging and its potential applications in various fields, including medical imaging.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: during the demosaicing process of hyperspectral images (HSI), how to achieve efficient and accurate reconstruction. Specifically, the research aims to develop and evaluate neural network architectures that can effectively process Multi - Spectral Filter Arrays (MSFAs) data, in order to achieve real - time applications and ensure high - quality spectral reconstruction. ### Problem Background Traditional hyperspectral image acquisition methods and devices have disadvantages such as high cost and long acquisition time. To overcome these challenges, researchers have proposed the method of using Multi - Spectral Filter Arrays (MSFAs), that is, performing spectral masking at the pixel level through a single sensor plane. However, this method will lead to a loss of spatial resolution and requires subsequent interpolation processing to recover the missing information. Common interpolation methods include bilinear interpolation, non - linear filtering, etc., but these methods perform poorly in handling complex scenes, especially prone to artifacts and spectral distortion in edge regions. ### Research Objectives The main contributions of this paper are: 1. **Propose an efficient neural network architecture**: Developed multiple new and improved demosaicing networks and compared them with classical interpolation methods and existing reference networks. 2. **Emphasize the accuracy of spectral reconstruction**: Not only focus on visual effects, but also pay more attention to the correct reconstruction of spectral information, which is crucial for application scenarios such as medical imaging. 3. **Achieve real - time applications**: By reducing the number of network parameters, improve computational efficiency, enabling the network to run on resource - limited real - time platforms. ### Main Results - **Network Architectures**: Proposed three different types of network architectures based on ResNet, U - Net and parallel structures. - **Datasets**: Constructed two datasets, "SimpleData" and "SimRealData", representing Multi - Spectral Filter Arrays data with different degrees of realism respectively. - **Experimental Results**: Experiments show that the proposed networks perform better than or match the existing reference models on the two datasets, especially in terms of spectral reconstruction. - **Practical Applications**: Demonstrated the demosaicing results of intraoperative snapshot images in surgery, verifying the effectiveness and practicality of the method. Through these works, the paper provides valuable insights for the practical application of hyperspectral imaging technology, especially in fields such as medical imaging, and promotes the integration and development of compact acquisition concepts such as snapshot mosaic imaging.

Efficient and Accurate Hyperspectral Image Demosaicing with Neural Network Architectures

PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Fast non-iterative blind restoration of hyperspectral images with PSFs

Efficient Training Procedures for Multi-Spectra Demosaicing

A Snapshot Near-Infrared Hyperspectral Demosaicing Method with Convolutional Neural Networks in Low Illumination Environment

An Efficient Transformer For Demosaicing Via Compressed Multi-Branch Attention Mechanism.

Multitask Sparse Representation Model-Inspired Network for Hyperspectral Image Denoising

Hyperspectral Imaging from a Raw Mosaic Image with End-to-end Learning

TV-enhanced Deep Unfolding Network for Multispectral Image Demosaicing

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging

A Snapshot Multi-Spectral Demosaicing Method for Multi-Spectral Filter Array Images Based on Channel Attention Network

Hyperspectral Image Reconstruction Based on Improved Residual Dense Network

Nonlocal Spatial–Spectral Neural Network for Hyperspectral Image Denoising

Deep Spatial Adaptive Network for Real Image Demosaicing.

MAC-Net: Model-Aided Nonlocal Neural Network for Hyperspectral Image Denoising.

Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

Hyperspectral demosaicking and crosstalk correction using deep learning

Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks

Channel-by-Channel Demosaicking Networks with Embedded Spectral Correlation

Hyperspectral Image Reconstruction Using Hierarchical Neural Architecture Search from A Snapshot Image.

Mask-Guided Spatial–Spectral MLP Network for High-Resolution Hyperspectral Image Reconstruction