Abstract:Object detection plays a crucial role in scene understanding and has extensive practical applications. In the field of remote sensing object detection, both detection accuracy and robustness are of significant concern. Existing methods heavily rely on sophisticated adversarial training strategies that tend to improve robustness at the expense of accuracy. However, detection robustness is not always indicative of improved accuracy. Therefore, in this article, we research how to enhance robustness, while still preserving high accuracy, or even improve both simultaneously, with simple vanilla adversarial training or even in the absence thereof. In pursuit of a solution, we first conduct an exploratory investigation by shifting our attention from adversarial training, referred to as adversarial fine-tuning, to adversarial pretraining. Specifically, we propose a novel pretraining paradigm, namely, structured adversarial self-supervised (SASS) pretraining, to strengthen both clean accuracy and adversarial robustness for object detection in remote sensing images. At a high level, SASS pretraining aims to unify adversarial learning and self-supervised learning into pretraining and encode structured knowledge into pretrained representations for powerful transferability to downstream detection. Moreover, to fully explore the inherent robustness of vision Transformers and facilitate their pretraining efficiency, by leveraging the recent masked image modeling (MIM) as the pretext task, we further instantiate SASS pretraining into a concise end-to-end framework, named structured adversarial MIM (SA-MIM). SA-MIM consists of two pivotal components: structured adversarial attack and structured MIM (S-MIM). The former establishes structured adversaries for the context of adversarial pretraining, while the latter introduces a structured local-sampling global-masking strategy to adapt to hierarchical encoder architectures. Comprehensive experiments on three different datasets have demonstrated the significant superiority of the proposed pretraining paradigm over previous counterparts for remote sensing object detection. More importantly, regardless of with or without adversarial fine-tuning, it enables simultaneous improvements in detection accuracy and robustness as expected, promisingly alleviating the dependence on complicated adversarial fine-tuning.

Using Multiple Self-Supervised Tasks Improves Model Robustness

Benchmarking Robust Self-Supervised Learning Across Diverse Downstream Tasks

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

On visual self-supervision and its effect on model robustness

Robust Remote Sensing Scene Classification by Adversarial Self-Supervised Learning

Improved Noise and Attack Robustness for Semantic Segmentation by Using Multi-Task Training with Self-Supervised Depth Estimation

Structured Adversarial Self-Supervised Learning for Robust Object Detection in Remote Sensing Images

Hyper Adversarial Tuning for Boosting Adversarial Robustness of Pretrained Large Vision Models

Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning

A Survey of Harnessing Self-Supervision Against Adversarial Attacks

On Inherent Adversarial Robustness of Active Vision Systems

Improving Adversarial Robustness via Decoupled Visual Representation Masking

Cross-Task Attack: A Self-Supervision Generative Framework Based on Attention Shift

THAT: Two Head Adversarial Training for Improving Robustness at Scale

Image Super-Resolution as a Defense Against Adversarial Attacks

Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks

Adversarial Example Defense Based on the Supervision

Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning.

Towards Adversarial Robustness with Multidimensional Perturbations Via Contrastive Learning

Enhancing Adversarial Robustness of Deep Neural Networks Through Supervised Contrastive Learning

A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models