Abstract:Thick cloud and cloud shadow often exist in optical remote sensing (RS) images, which limit the application range of RS data. To solve this challenging problem, we propose a multitemporal-based method of cloud removal to reconstruct the large cloud-cover region in RS images. The method can combine the advantages of high accuracy of deep-learning method and low computation cost of traditional method, and reduce the difficulty of establishing multi-temporal dataset to a certain extent. In the proposed method, the complex problem of thick cloud removal is decomposed into two simple processes: coarse stage and refined stage. In the stage of coarse cloud removal, it is found that the cloud-cover images are distributed more regularly in frequency domain than in the space domain. Therefore, the frequency domain residual module is introduced, and the frequency domain attention module is designed to achieve more focused fusion of three-temporal features. In the stage of refined cloud removal, the refined network is designed to retain the effective features of the coarse stage and further refine the reconstruction result based on the image self-correlation. We carried out sufficient experiments to demonstrate the superiority of the proposed method compared with four classical methods, which include WLR, STSCNN, RFRnet and PSTCR. In qualitative experiments, the proposed method has obvious advantages in information accuracy, visual naturalness and semantic rationality. In quantitative experiments, compared with the best of four comparison methods, average values of Peak Signal to Noise Ratio (PSNR) on three-temporal images are improved by 5.1934 dB, 2.7729 dB and 7.2364 dB respectively, and average values of Correlation Coefficient (CC) on three-temporal images are increased by 0.1048, 0.0110 and 0.0966 respectively. Moreover, the proposed method has an obvious advantage in computation cost because it can reconstruct multiple cloud-cover images synchronously.

Inpainting of Remote Sensing SST Images With Deep Convolutional Generative Adversarial Network.

Image Inpainting Based on Interactive Separation Network and Progressive Reconstruction Algorithm

Remote sensing image cloud removal by deep image prior with a multitemporal constraint

Panoramic Image Inpainting With Gated Convolution And Contextual Reconstruction Loss

Cloud-Aware Generative Network: Removing Cloud From Optical Remote Sensing Images

Gated Convolutional Networks for Cloud Removal From Bi-Temporal Remote Sensing Images

A deep-learning reconstruction method for remote sensing images with large thick cloud cover

Inpainting with Separable Mask Update Convolution Network

Coarse-to-fine Task-driven Inpainting for Geoscience Images

An Innovative Approach for Effective Removal of Thin Clouds in Optical Images Using Convolutional Matting Model

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network

Context-Based Multiscale Unified Network for Missing Data Reconstruction in Remote Sensing Images

Cloud Removal with Fusion of High Resolution Optical and SAR Images Using Generative Adversarial Networks

RSINet: Inpainting Remotely Sensed Images Using Triple GAN Framework

Thick Cloud Removal in Optical Remote Sensing Images Using a Texture Complexity Guided Self-Paced Learning Method

Cloud Removal for Remote Sensing Imagery via Spatial Attention Generative Adversarial Network

Thin Cloud Removal Generative Adversarial Network Based on Sparse Transformer in Remote Sensing Images

An Adaptive Iterative Inpainting Method with More Information Exploration

Thick cloud and cloud shadow removal in multitemporal imagery using progressively spatio-temporal patch group deep learning