Abstract:Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

An efficient CNN for spectral reconstruction from RGB images

In Defense of Shallow Learned Spectral Reconstruction from RGB Images

HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images

Cluster-based deep convolutional networks for spectral reconstruction from RGB images

U-Net-Based Multispectral Image Generation from an RGB Image

Spectral Image Reconstruction Using Recovered Basis Vector Coefficients

Residual HSRCNN: Residual Hyper-Spectral Reconstruction CNN from an RGB Image

Transfer Learning for Spectral Image Reconstruction from RGB Images

Physically Plausible Spectral Reconstruction from RGB Images

Hsvcnn: Cnn-Based Hyperspectral Reconstruction From Rgb Videos

Efficient Transfer Learning for Spectral Image Reconstruction from RGB Images

RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN

Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images

Random Convolutional Network for Hyperspectral Image Classification.

Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Deep-learning-based hyperspectral imaging through a RGB camera

Cross Spectral Image Reconstruction Using a Deep Guided Neural Network

Spectral super-resolution meets deep learning: Achievements and challenges