Abstract:This paper formulates cloud and cloud shadow detection as a semantic segmentation problem and proposes a deep convolutional neural network (CNN) based method to detect them in Landsat imagery. Different from traditional machine learning methods, deep CNN-based methods convolve the entire input image to extract multi-level spatial and spectral features, and then deconvolve these features to produce the detailed segmentation. In this way, multi-level features from the whole image and all the bands are utilized to label each pixel as cloud, thin cloud, cloud shadow or clear. An adaption of SegNet with 13 convolutional layers and 13 deconvolution layers is proposed in this study. The method is applied to 38 Landsat 7 images and 32 Landsat 8 images which are globally distributed and have pixel-wise cloud and cloud shadow reference masks provided by the U.S. Geological Survey (USGS). In order to process such large images using the adapted SegNet model on a desktop computer, the Landsat Collection 1 scenes are split into non-overlapping 512 * 512 30 m pixel image blocks. 60% of these blocks are used to train the model using the backpropagation algorithm, 10% of the blocks are used to validate the model and tune its parameters, and the remaining 30% of the blocks are used for performance evaluation. Compared with the cloud and cloud shadow masks produced by CFMask, which are provided with the Landsat Collection 1 data, the overall accuracies are significantly improved from 89.88% and 84.58% to 95.26% and 95.47% for the Landsat 7 and Landsat 8 images respectively. The proposed method benefits from the multi-level spatial and spectral features, and results in more than a 40% increase in user's accuracy and in more than a 20% increase in producer's accuracy for cloud shadow detection in Landsat 8 imagery. The issues for operational implementation are discussed.

Multi-label Cloud Segmentation Using a Deep Network

CloudSegNet: A Deep Network for Nychthemeron Cloud Image Segmentation

CloudU-Netv2: A Cloud Segmentation Method for Ground-Based Cloud Images Based on Deep Learning

SegCloud: a Novel Cloud Image Segmentation Model Using a Deep Convolutional Neural Network for Ground-Based All-Sky-view Camera Observation

CloudY-Net: A Deep Convolutional Neural Network Architecture for Joint Segmentation and Classification of Ground-Based Cloud Images

A Novel Ground-Based Cloud Image Segmentation Method by Using Deep Transfer Learning

CloudU-Net: A Deep Convolutional Neural Network Architecture for Daytime and Nighttime Cloud Images’ Segmentation

CloudDeepLabV3+: a lightweight ground-based cloud segmentation method based on multi-scale feature aggregation and multi-level attention feature enhancement

Automatic Cloud Segmentation Based on Fused Fully Convolutional Networks.

Cloud Segmentation and Matching Using Deep Learning in All-Sky Images

An Aerial Image Segmentation Approach Based on Enhanced Multi-Scale Convolutional Neural Network

Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks

ACLNet: An Attention and Clustering-based Cloud Segmentation Network

End-to-end Cloud Segmentation in High-Resolution Multispectral Satellite Imagery Using Deep Learning

Multi-scale convolutional neural networks for cloud segmentation

Dual-Branch Network for Cloud and Cloud Shadow Segmentation

Cloud/shadow segmentation based on multi-level feature enhanced network for remote sensing imagery

Multi-Branch Attention Fusion Network for Cloud and Cloud Shadow Segmentation

TransCloudSeg: Ground-Based Cloud Image Segmentation with Transformer

Multipath Multiscale Attention Network for Cloud and Cloud Shadow Segmentation

Annotated Dataset for Training Cloud Segmentation Neural Networks Using High-Resolution Satellite Remote Sensing Imagery