Abstract:Deep learning methods have been proven effective in remote sensing image analysis and interpretation, where semantic segmentation plays a vital role. These deep segmentation methods are susceptible to adversarial attacks, while most of the existing attack methods tend to manipulate the image globally, leading to noticeable perturbations and chaotic segmentation. In this work, we propose a novel stealthy attack for semantic segmentation (SASS), which can largely increase the effectiveness and stealthiness from the existing attack methods on remote sensing images. SASS manipulates specific victim classes or objects of interest while preserving the original segmentation results for other classes or objects. In practice, as different inference mechanisms, overlapped inference, can be applied in segmentation, the efficacy of SASS may be degraded. To this end, we further introduce the masked SASS (MSASS), which generates augmented adversarial perturbations that only affect victim areas. We evaluate the effectiveness of SASS and MSASS using four state-of-the-art semantic segmentation models on the Vaihingen and Zurich Summer datasets. Extensive experiments demonstrate that our SASS and MSASS methods achieve superior attack performances on victim areas while maintaining high accuracies of other areas (drop less than 2%). The detection success rates of adversarial examples for segmentation, as characterized by Xiao et al., significantly drop from 97.78% for the untargeted projected gradient descent (PGD) attack to 28.71% for our MSASS method on the Zurich Summer dataset. Our work contributes to the field of adversarial attacks in semantic segmentation for remote sensing images by improving stealthiness, flexibility, and robustness. We anticipate that our findings will inspire the development of defense methods to enhance the security and reliability of semantic segmentation models against our stealthy attack.

Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models

Evaluating the Adversarial Robustness of Semantic Segmentation: Trying Harder Pays Off

On the Robustness of Semantic Segmentation Models to Adversarial Attacks

In Defense Of Multi-Source Omni-Supervised Efficient Convnet For Robust Semantic Segmentation In Heterogeneous Unseen Domains

Semantically Consistent Visual Representation for Adversarial Robustness

Stealthy Adversarial Examples for Semantic Segmentation in Remote Sensing

Learning to Generate Training Datasets for Robust Semantic Segmentation

Improving adversarial robustness of deep neural networks by using semantic information

Towards Robust General Medical Image Segmentation

Detecting Adversarial Attacks in Semantic Segmentation via Uncertainty Estimation: A Deep Analysis

Benchmarking the Robustness of Semantic Segmentation Models with Respect to Common Corruptions

Single-Step Adversarial Training for Semantic Segmentation

Analyzing Adversarial Robustness of Deep Neural Networks in Pixel Space: a Semantic Perspective

On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving

Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks

On Adversarial Robustness of Point Cloud Semantic Segmentation

Towards Adversarial Robustness of Model-Level Mixture-of-Experts Architectures for Semantic Segmentation

Robustifying Deep Networks for Medical Image Segmentation

Robustness Exploration of Semantic Information in Adversarial Training

Evaluating the Robustness of Off-Road Autonomous Driving Segmentation against Adversarial Attacks: A Dataset-Centric analysis

Improved Noise and Attack Robustness for Semantic Segmentation by Using Multi-Task Training with Self-Supervised Depth Estimation