Abstract:Cloud and cloud shadow segmentation is a crucial pre-processing step for any application that uses multi spectral satellite images. In particular, disaster related applications (e.g., flood monitoring or rapid damage mapping), which are highly time- and data-critical, require methods that produce accurate cloud and cloud shadow masks in short time while being able to adapt to large variations in the target domain (induced by atmospheric conditions, different sensors, scene properties, etc.). In this study, we propose a data-driven approach to semantic segmentation of cloud and cloud shadow in single date images based on a modified U-Net convolutional neural network that aims to fulfil these requirements. We train the network on a global database of Landsat OLI images for the segmentation of five classes ("shadow", "cloud", "water", "land" and "snow/ice"). We compare the results to state-of-the-art methods, proof the model's generalization ability across multiple satellite sensors (Landsat TM, Landsat ETM+, Landsat OLI and Sentinel-2) and show the influence of different training strategies and spectral band combinations on the performance of the segmentation. Our method consistently outperforms Fmask and a traditional Random Forest classifier on a globally distributed multi-sensor test dataset in terms of accuracy, Cohen's Kappa coefficient, Dice coefficient and inference speed. The results indicate that a reduced feature space composed solely of red, green, blue and near-infrared bands already produces good results for all tested sensors. If available, adding shortwave-infrared bands can increase the accuracy. Contrast and brightness augmentations of the training data further improve the segmentation performance. The best performing U-Net model achieves an accuracy of 0.89, Kappa of 0.82 and Dice coefficient of 0.85, while running the inference over 896 test image tiles with 44.8 s/megapixel (2.8 s/megapixel on GPU). The Random Forest classifier reaches an accuracy of 0.79, Kappa of 0.65 and Dice coefficient of 0.74 with 3.9 s/megapixel inference time (on CPU) on the same training and testing data. The rule-based Fmask method takes significantly longer (277.8 s/megapixel) and produces results with an accuracy of 0.75, Kappa of 0.60 and Dice coefficient of 0.72.

MRFA-Net: Multi-Scale Receptive Feature Aggregation Network for Cloud and Shadow Detection

Multi-Branch Attention Fusion Network for Cloud and Cloud Shadow Segmentation

MSFANet: Multi-Scale Strip Feature Attention Network for Cloud and Cloud Shadow Segmentation

Multiscale Attention Feature Aggregation Network for Cloud and Cloud Shadow Segmentation

AFMUNet: Attention Feature Fusion Network Based on a U-Shaped Structure for Cloud and Cloud Shadow Detection

Detail-Aware Multiscale Context Fusion Network for Cloud Detection

MRPFA-Net for Shadow Detection in Remote-Sensing Images

Multipath Multiscale Attention Network for Cloud and Cloud Shadow Segmentation

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Cloud/shadow segmentation based on multi-level feature enhanced network for remote sensing imagery

CRSNet: Cloud and Cloud Shadow Refinement Segmentation Networks for Remote Sensing Imagery

Multi-Stage Frequency Attention Network for Progressive Optical Remote Sensing Cloud Removal

Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors

MFCANet: Multiscale Feature Context Aggregation Network for Oriented Object Detection in Remote-Sensing Images

Effective Detection of Cloud Masks in Remote Sensing Images

MFALNet: A Multiscale Feature Aggregation Lightweight Network for Semantic Segmentation of High-Resolution Remote Sensing Images.

Multi-sensor Cloud and Cloud Shadow Segmentation with a Convolutional Neural Network

A Unified Cloud Detection Method for Suomi-NPP VIIRS Day and Night PAN Imagery

MRF-Net: An Infrared Remote Sensing Image Thin Cloud Removal Method With the Intra-Inter Coherent Constraint

Radiation Feature Fusion Dual-Attention Cloud Segmentation Network

Multi-View Feature Fusion and Rich Information Refinement Network for Semantic Segmentation of Remote Sensing Images