Simultaneous Cloud Detection and Removal From Bitemporal Remote Sensing Images Using Cascade Convolutional Neural Networks

Shunping Ji,Peiyu Dai,Meng Lu,Yongjun Zhang

DOI: https://doi.org/10.1109/tgrs.2020.2994349

IF: 8.2

2021-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Clouds and cloud shadows heavily affect the quality of the remote sensing images and their application potential. Algorithms have been developed for detecting, removing, and reconstructing the shaded regions with the information from the neighboring pixels or multisource data. In this article, we propose an integrated cloud detection and removal framework using cascade convolutional neural networks, which provides accurate cloud and shadow masks and repaired images. First, a novel fully convolutional network (FCN), embedded with multiscale aggregation and the channel-attention mechanism, is developed for detecting clouds and shadows from a cloudy image. Second, another FCN, with the masks of the detected cloud and shadow, the cloudy image, and a temporal image as the input, is used for the cloud removal and missing-information reconstruction. The reconstruction is realized through a self-training strategy that is designed to learn the mapping between the clean-pixel pairs of the bitemporal images, which bypasses the high demand of manual labels. Experiments showed that our proposed framework can simultaneously detect and remove the clouds and shadows from the images and the detection accuracy surpassed several recent cloud-detection methods; the effects of image restoring outperform the mainstream methods in every indicator by a large margin. The data set used for cloud detection and removal is made open.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics

What problem does this paper attempt to address?

The paper attempts to address the problem of simultaneously detecting and removing clouds and cloud shadows in remote sensing images. Specifically:

- **Research Background**: Clouds and cloud shadows severely affect the quality and application potential of remote sensing images. Currently, various algorithms exist for detecting, removing clouds, and reconstructing shadow areas, but these methods usually handle these two related tasks independently.
- **Main Objective**: To propose an integrated framework that utilizes a cascaded convolutional neural network (CNN) to achieve a unified process for cloud detection and removal. This approach not only improves detection accuracy but also better restores information in occluded areas.
- **Innovations**:
- A new fully convolutional network (FCN) structure is proposed, combining multi-scale aggregation and channel attention mechanisms to detect clouds and cloud shadows.
- A self-training strategy is used to train another FCN to remove clouds, which can learn the mapping relationship between temporal images without manually labeled samples.
- Experiments show that the proposed framework outperforms existing mainstream methods in both cloud detection and image restoration.
- **Dataset**: To validate the effectiveness of the method, the researchers created an open dataset (WHU Cloud Dataset), which includes remote sensing image pairs from multiple different regions and conducted detailed experimental analysis.

In summary, the paper aims to address the limitations of traditional methods that handle cloud detection and removal separately by using a comprehensive deep learning framework and demonstrates its superior performance in practical applications.