Abstract:Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

SR-LLA: A novel spectral reconstruction method based on locally linear approximation

Spectral-information-divergence based local pls modeling algorithm in near infrared spectroscopy

Source Reconstruction Method Based on Machine Learning Algorithms

Locally Linear Reconstruction for Spectral Enhancement Using Limited Pixel-to-Pixel Multispectral and Hyperspectral Data

Local Spectral Similarity Preserving Regularized Robust Sparse Hyperspectral Unmixing

Robust Tensor Low-Rank Sparse Representation with Saliency Prior for Hyperspectral Anomaly Detection

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Unsupervised spectral reconstruction from RGB images under two lighting conditions

SR-SIM: A Fast and High Performance IQA Index Based on Spectral Residual

A Sparse Classification Based on a Linear Regression Method for Spectral Recognition

Research on the deep learning-based exposure invariant spectral reconstruction method

Salient Object Detection Employing Robust Sparse Representation and Local Consistency

Spectral Super-Resolution via Deep Low-Rank Tensor Representation

Spectral Reconstruction Network From Multispectral Images to Hyperspectral Images: A Multitemporal Case

Subspace And Sparse Reconstruction Based Near-Field Sources Localization In Uniform Linear Array

Spatial Spectrum Estimation Algorithm based on Minimum Redundancy Linear Array: A Performance Analysis

DLRA-Net: Deep Local Residual Attention Network with Contextual Refinement for Spectral Super-Resolution

Fast Non-Local Neural Networks with Spectral Residual Learning

Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses

Spectral-Spatial Hyperspectral Image Classification Via Locality and Structure Constrained Low-Rank Representation

Spectral Reconstruction for Paired Images Based on Semi-supervised Deep Learning