LatentHSI: Restore hyperspectral images in a latent space

Jin Cao,Xiangyu Rui,Li Pang,Deyu Meng,Xiangyong Cao
DOI: https://doi.org/10.1016/j.inffus.2024.102848
IF: 18.6
2024-12-10
Information Fusion
Abstract:Hyperspectral image (HSI) restoration is a critical task for remote sensing. The high dimensionality of HSIs poses significant challenges for restoration techniques. Existing methods often involve transforming the image into a lower-dimensional space to address the problem. However, most of these constructed spaces either require handcrafted prior assumptions or ignore utilizing the prior knowledge of HSIs. In this work, we propose a general unsupervised HSI restoration approach in which a new lower-dimensional latent space that fully considers the prior of HSIs is learned automatically from the HSIs. Specifically, we first decompose a clean HSI into the product of a reduced representation and a coefficient matrix by exploiting the low-rank property inherent in HSIs. Then, we construct a latent space for the reduced representation using Variational Autoencoder (VAE) and perform sampling in this space via a latent diffusion model guided by a tailored guidance function. Finally, a correction step is further used to refine the restored image produced by the constructed latent space. Experimental results demonstrate that our method can achieve state-of-the-art (SOTA) performance, surpassing existing methods in both quantitative metrics and visual quality on various HSI restoration tasks, e.g., pansharpening, HSI denoising, and noisy HSI super-resolution.
computer science, artificial intelligence, theory & methods
What problem does this paper attempt to address?