Estimating Uncertainty in Landslide Segmentation Models

Savinay Nagendra,Chaopeng Shen,Daniel Kifer

2024-03-26

Abstract:Landslides are a recurring, widespread hazard. Preparation and mitigation efforts can be aided by a high-quality, large-scale dataset that covers global at-risk areas. Such a dataset currently does not exist and is impossible to construct manually. Recent automated efforts focus on deep learning models for landslide segmentation (pixel labeling) from satellite imagery. However, it is also important to characterize the uncertainty or confidence levels of such segmentations. Accurate and robust uncertainty estimates can enable low-cost (in terms of manual labor) oversight of auto-generated landslide databases to resolve errors, identify hard negative examples, and increase the size of labeled training data. In this paper, we evaluate several methods for assessing pixel-level uncertainty of the segmentation. Three methods that do not require architectural changes were compared, including Pre-Threshold activations, Monte-Carlo Dropout and Test-Time Augmentation -- a method that measures the robustness of predictions in the face of data augmentation. Experimentally, the quality of the latter method was consistently higher than the others across a variety of models and metrics in our dataset.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the following issues: 1. **Uncertainty Assessment in Landslide Segmentation Models**: The paper primarily focuses on identifying landslide areas from satellite images and explores methods to assess the uncertainty of landslide segmentation models at the pixel level. Accurate uncertainty estimation helps reduce the workload of manual verification and improves the quality of automatically annotated databases. 2. **Landslide Database Construction**: Since manually annotating landslide datasets is time-consuming and prone to errors, the paper proposes a method to construct large-scale landslide databases through automated and semi-automated approaches, emphasizing the importance of accurately assessing model prediction confidence. Specifically, the authors compared three methods (without changing the model architecture) to measure the uncertainty of each pixel: - Pre-Threshold Values - Monte-Carlo Dropout - Test-Time Augmentation Experimental results show that the Test-Time Augmentation (TTA) method outperforms the other two methods across various models and evaluation metrics. Additionally, the paper provides a detailed description of the dataset construction process, performance comparison of different models, and evaluation criteria for uncertainty quantification techniques.

Estimating Uncertainty in Landslide Segmentation Models

Give me (un)certainty -- An exploration of parameters that affect segmentation uncertainty

Mitigating Distributional Shift in Semantic Segmentation via Uncertainty Estimation from Unlabelled Data

Mitigating Distributional Shift in Semantic Segmentation via Uncertainty Estimation From Unlabeled Data

Segmentation Re-thinking Uncertainty Estimation Metrics for Semantic Segmentation

Uncertainty estimation in Deep Learning for Panoptic segmentation

ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation

Hyperbolic Uncertainty Aware Semantic Segmentation

Probabilistic Deep Learning for Instance Segmentation

On the Calibration of Uncertainty Estimation in LiDAR-based Semantic Segmentation

Landslide susceptibility prediction based on image semantic segmentation

Landslide Segmentation with U-Net: Evaluating Different Sampling Methods and Patch Sizes

Laplacian Segmentation Networks Improve Epistemic Uncertainty Quantification

Uncertainty Quantification for Bird's Eye View Semantic Segmentation: Methods and Benchmarks

Reliability in Semantic Segmentation: Are We on the Right Track?

Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation

Uncertainty Quantification in Computer-Aided Diagnosis: Make Your Model say "I don't know" for Ambiguous Cases

Estimating Uncertainty of Geographic Atrophy Segmentations with Bayesian Deep Learning

Uncertainty estimates for semantic segmentation: providing enhanced reliability for automated motor claims handling

Learning to segment subcortical structures from noisy annotations with a novel uncertainty-reliability aware learning framework

Uncertainty Estimation and Out-of-Distribution Detection for LiDAR Scene Semantic Segmentation