Abstract:Multispectral image (MSI) is essential for remote sensing (RS) tasks. Unfortunately, the presence of clouds will lead to a reduction in the accuracy and availability of ground information. Synthetic aperture radar (SAR) is unaffected by cloud cover, which can serve as a supplemental information source to alleviate the difficulty of cloud removal in MSI. However, SAR images suffer from the problem of speckle noise and contain information only in a single spatial dimension. To achieve cloud removal effectively by fusing MSI and SAR, we propose the TransFusion-CR in this article. It is a novel two-phase SAR and MSI translation-fusion cloud removal network and designed to enhance the information gains provided by SAR. We use a convolutional neural network (CNN) with a large kernel and design special modules to process the global and local information so that TransFusion-CR can reconstruct clear spatial details and accurate spectral information without artifacts. In the translation phase, we establish an SAR frequency extraction (SFE) module based on the discrete wavelet transform (DWT) to separate and process different frequency information of the SAR image. This assists in dealing with the high-frequency speckle noise while generating translated cloud-free MSI with clear spatial information and reliable spectral information. Subsequently, in the fusion phase, a specially designed feature fusion module realizes a weighted triple-source image fusion guided by the cloud mask score (CMS) image, which high-quality cloud-free MSI. Finally, a new loss function is proposed to balance the learning focus and achieve joint optimization. Extensive experiments demonstrate that TransFusion-CR outperforms the state-of-the-art by about 1.3 dB in terms of PSNR when the percentage of thick cloud cover is more than 60% and can construct reliable terrestrial information even in the complete absence of surface data.

Cloud Removal Based on SAR-Optical Remote Sensing Data Fusion via a Two-Flow Network

Algorithm for cloud removal in optical-synthetic aperture radar fusion images based on improved residual networks

Cloud Removal with Fusion of High Resolution Optical and SAR Images Using Generative Adversarial Networks

Sentinel-2 Cloud Removal Considering Ground Changes by Fusing Multitemporal SAR and Optical Images

Cloud Removal With SAR-Optical Data Fusion Using a Unified Spatial–Spectral Residual Network

An Innovative Approach for Effective Removal of Thin Clouds in Optical Images Using Convolutional Matting Model

Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion

Efficient Cloud Removal Network for Satellite Images Using SAR-Optical Image Fusion

LGE-CR: Integrating Nonlocal Gated Attention and Two-Flow Residual Dense Learning for Cloud Removal

A New Cloud Removal Algorithm for Multi-Spectral Images

Denoising Diffusion Probabilistic Feature-Based Network for Cloud Removal in Sentinel-2 Imagery

Multi-Stage Frequency Attention Network for Progressive Optical Remote Sensing Cloud Removal

A Spatiotemporal Fusion Based Cloud Removal Method for Remote Sensing Images With Land Cover Changes

TransFusion-CR: Two-Phase SAR-to-Optical Translation and Deep Feature Fusion for Cloud Removal

Remote sensing image cloud removal based on multi-scale spatial information perception

Spatio-Temporal SAR-Optical Data Fusion for Cloud Removal via a Deep Hierarchical Model

Thick Cloud Removal in Multitemporal Remote Sensing Images Using a Coarse-to-Fine Framework

Gated Convolutional Networks for Cloud Removal From Bi-Temporal Remote Sensing Images

Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery

CERMF-Net: A SAR-Optical Feature Fusion for Cloud Elimination From Sentinel-2 Imagery Using Residual Multiscale Dilated Network