Collaborative Dual-Harmonization Reconstruction Network for Large-Ratio Cloud Occlusion Missing Information in High-Resolution Remote Sensing Images
Weiling Liu,Yonghua Jiang,Fangting Li,Guo Zhang,Huaibo Song,Chengjun Wang,Xinghua Li
DOI: https://doi.org/10.1016/j.engappai.2024.108861
IF: 8
2024-01-01
Engineering Applications of Artificial Intelligence
Abstract:High -resolution optical remote sensing images are the primary source of data for fine Earth observations. However, extensive cloud occlusion often limits the immediate use of such images. Moreover, although the reconstructed and composite images maintain consistent semantics and appearance in the non -reconstructed regions, they can present structural similarities and spectral variances in the reconstructed regions. Existing models use undifferentiated feature extraction for the reconstructed and non -reconstructed regions of reconstructed and composite images, exacerbating problems (such as the boundary effect) and affecting the overall consistency in the reconstructed images. Consequently, we designed a collaborative dual -harmonization reconstruction network, namely, the CDH-RecNet network, from the perspective of pixel -level local context and global -level context and proposed a pixel -to -pixel fine-grained local harmonization module (PFLH), image -to -image global harmonization module (IGH), ground constraint module (GCM), and foreground- background aware module (FBAM) for reconstructed and non -reconstructed regions to achieve the goal of reconstructing large ratios of missing information in high -resolution images. Compared to the classical recurrent feature reasoning network (RFR-Net), spatiotemporal generator network (STGAN), spatial-temporal- spectral convolutional neural network (STS -CNN), and bishift network (BSN), our model can be applied to the problem of reconstructing missing information with different resolutions, from different satellites, with different band combinations, addresses different missing types and different cloud occlusion distributions, and is not affected by cloud density and distribution status. Specifically, (1) our model shows significant advantages in 40%, 50%, and 70% large -ratio cloud occlusion reconstruction in the high -resolution image Gaofen-1(2 m), and the clarity index is improved by 1.2599, 1.2169, and 1.0678 at 40 %, by 1.4392, 1.1277, and 0.9245 at 50%; by 2.5493, 2.3635, and by 2.1776 at 70%; (2) it also well -reconstructs 40%, 50%, and 70% large -ratio cloud -occluded information in Sentinel -2 (10 m), Gaofen-1 (16 m), and Landsat -8 (30 m) medium and lowresolution satellite images with different band combinations; (3) our model has an excellent reconstruction effect on Landsat -7 (30 m) 20% and 40% stripes missing.