Compressing Hyperspectral Images Into Multilayer Perceptrons Using Fast-Time Hyperspectral Neural Radiance Fields
Lili Zhang,Tianpeng Pan,Jiahui Liu,Lin Han
DOI: https://doi.org/10.1109/lgrs.2024.3365106
IF: 5.343
2024-02-23
IEEE Geoscience and Remote Sensing Letters
Abstract:Hyperspectral images play an important role in the field of remote sensing (RS), and similar to ordinary RGB-based images, reconstructing the original information with higher quality using fewer bits is an essential task. Most existing hyperspectral image compression methods utilize a transform-based compression framework that reconstructs the original image after converting the input into a latent representation with a specific size. This kind of approaches has achieved some success, and however, it suffers from two problems. First, the encoder and decoder used for transformation take up a huge amount of computational resources, both for training and deployment. Second, the upper performance limit is not satisfactory, that is to say, huge computational cost does not bring a matching performance gain. Based on this, we propose a novel hyperspectral image compression method. Specifically, we employ neural radiance fields (NeRFs) to compress hyperspectral images, and unlike transform-based methods, the proposed method encodes the hyperspectral coordinate information, which is fit to hyperspectral pixel values using multilayer perceptrons (MLPs). After that, we only need to compress the weights of the generated MLPs using the model compression method (in this letter, we only employ the weight quantization) to efficiently save the hyperspectral images since the MLPs model, working as the fitting function, can be regarded as the compressed representation of the hyperspectral image. At the same condition, the proposed method achieved nearly 5 dB higher in PSNR and 6 dB higher in MS-SSIM than the comparison deep-learning-based methods.
imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics