Generalized Spectral Super-resolution for Multispectral Satellite Imagings
He Jiang,Yuan Qiangqiang,Li Jie
DOI: https://doi.org/10.3788/gzxb20235202.0210002
IF: 0.6
2023-01-01
ACTA PHOTONICA SINICA
Abstract:Spectral super-resolution, a very important computational imaging technology to obtain high spatial-resolution hyperspectral images at a tow cost, has received more and more attention. However, the existing works about spectral super-resolution are all based on the assumption that there is only spectral degradation between the observed multispectral data and the real spectra. For multispectral satellites, different imaging modes also include spatial degradation. For this type of data, the existing spectral super-resolution usually uses only a part of the multispectral data to reconstruct hyperspectral data, which will lead to the waste of multispectral data. This paper extends the traditional spectral super resolution to the generalized spectral super resolution by summarizing the imaging modes of different multispectral satellites. There are FusSR, which makes full use of the additional nmitispectral hands with a lower spatial resolution to further optimize the spectral reconstruction, and PansSR, which uses the panchromatic channel with a higher spatial resolution to simultaneously improve the spatial resolution of the reconstructed hyperspectral data. The above two extended spectral super resolution technologies have made the best of all multispectral data. Besides, after modeling the imaging degradation, the generalized spectral super resolution is expressed as an optimal problem containing two data fidelity terms and one image prior term. To ensure the algorithm's physical interpretability, the deep unrolling strategy is adopted to build a generalized spectral super-resolution network that combines data-driven and model-driven manner. In addition, the idea of spectral grouping is also employed to generate the initial results. The spectral grouping includes three steps. Firstly, the difference information between bands is calculated. Then the coverage relationship of spectral response function between hyperspectral images and multispectral images is counted. Lastly, bands with high correlation are uniformly reconstructed, so as to eliminate mutual interference between bands with a large radiation gap. To discuss the feasibility of combining model driven and data driven algorithms in the generalized spectral super resolution problem, this paper proposed multiple multispectral satellite data sets, named Sen2OHS and GF2Hyper respectively. The former includes four high-resolution Sentinel 2 multispectral hands, four low-resolution Sentinel-2 multispectral bands, and 32 high resolution Zhuhai-1 hyperspectral bands; the latter includes four low resolution nmitispectral bands, one high-resolution panchromatic hand and 63 high-resolution hyperspectral hands. CC, mPSNR, mSSIM and SAM are used to evaluate the reconstruction quality. Comparing the results of traditional sSR and FusSR, we can find that the quantitative result of FusSR is higher than sSR. It can be inferred that introducing additional spectral information can effectively improve the spectral reconstruction results, even if they are low spatial resolution. Comparing the data before and after PansSR, we can see that not only the spectral channel number of the input data has increased, but also its spatial resolution has been improved. Above resulets show that using higher resolution panchannatic data can effectively help spectral super resolution improve spatial resolution. Whether in FusSR or PansSR, their experimental results in this paper effectively prove that a broader concept of spectral super-resolution should he proposed for remote sensing satellite data to effectively reduce data waste.