Abstract:Hyperspectral videos (HSVs) play an important role in the monitoring domain, as they can provide more information than red–green–blue (RGB) videos about the movement of interesting objects from the perspective of material interpretation. However, the acquisition of HSV data is expensive and time-consuming, whereas RGB videos are readily available. In order to obtain HSV data from its corresponding RGB data, this article proposes a lightweight frequency-spectrum unfolding network (FSUF-Net) for spectral super-resolution (SSR) of RGB video data. Specifically, the proposed FSUF-Net method belongs to a data-knowledge-driven joint paradigm, which is an interpretable SSR model instead of an end-to-end black-box architecture. The FSUF-Net consists of five main steps. First, the conversion representation of RGB video data to HSV data is derived into an initial recovery term, a data term, and a prior term according to a variable splitting method. Second, the spectral response function between hyperspectral images (HSIs) and RGB images is utilized to achieve the initial recovery term. Third, a convolutional neural network (CNN)-based frequency-domain subnetwork (F-Net) is designed to solve the data subproblem for recovering the spatial detail information from the HSI, and a Transformer-based spectrum-domain subnetwork (S-Net) is developed to solve the prior subproblem for reconstructing the spectral information of the HSI. Fourth, two network modules are employed to conduct parametric self-learning. Finally, the HSV data can be obtained in a fixed number of iterations, including alternately solving the above data subproblem and the prior subproblem. Experiments performed on several real datasets demonstrated that the FSUF-Net can effectively reconstruct HSV from RGB videos compared to traditional and state-of-the-art SSR methods. The proposed method is available online: https://github.com/chengle-zhou/HSV-SSR_FSUF-Net.

Multi-scale HSV Color Feature Embedding for High-fidelity NIR-to-RGB Spectrum Translation

Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation

U-Net-Based Multispectral Image Generation from an RGB Image

A Multi-Model Fusion Framework for NIR-to-RGB Translation.

Lightweight Multiresolution Feature Fusion Network for Spectral Super-Resolution

ColorMamba: Towards High-quality NIR-to-RGB Spectral Translation with Mamba

Attention-Guided NIR Image Colorization via Adaptive Fusion of Semantic and Texture Clues

Multi-Spectral RGB-NIR Image Classification Using Double-Channel CNN

Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translation

LTM-NeRF: Embedding 3D Local Tone Mapping in HDR Neural Radiance Field

Multispectral Fusion of RGB and NIR Images Using Weighted Least Squares and Convolution Neural Networks

RGB-to-HSV: A Frequency-Spectrum Unfolding Network for Spectral Super-Resolution of RGB Videos

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Near-Infrared Image Colorization Using Asymmetric Codec and Pixel-Level Fusion

Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising

A robust infrared and visible image fusion framework via multi-receptive-field attention and color visual perception

HDECGCN: A Heterogeneous Dual Enhanced Network Based on Hybrid CNNs Joint Multiscale Dynamic GCNs for Hyperspectral Image Classification

Spectral Segmentation Multi-Scale Feature Extraction Residual Networks for Hyperspectral Image Classification

Dual-Branch Colorization Network for Unpaired Infrared Images Based on High-Level Semantic Features and Multiscale Residual Attention

RGB-Induced Feature Modulation Network for Hyperspectral Image Super-Resolution.

Spectral Super-Resolution via Model-Guided Cross-Fusion Network