Abstract:New transformer networks have been integrated into object tracking pipelines and have demonstrated strong performance on the latest benchmarks. This paper focuses on understanding how transformer trackers behave under adversarial attacks and how different attacks perform on tracking datasets as their parameters change. We conducted a series of experiments to evaluate the effectiveness of existing adversarial attacks on object trackers with transformer and non-transformer backbones. We experimented on 7 different trackers, including 3 that are transformer-based, and 4 which leverage other architectures. These trackers are tested against 4 recent attack methods to assess their performance and robustness on VOT2022ST, UAV123 and GOT10k datasets. Our empirical study focuses on evaluating adversarial robustness of object trackers based on bounding box versus binary mask predictions, and attack methods at different levels of perturbations. Interestingly, our study found that altering the perturbation level may not significantly affect the overall object tracking results after the attack. Similarly, the sparsity and imperceptibility of the attack perturbations may remain stable against perturbation level shifts. By applying a specific attack on all transformer trackers, we show that new transformer trackers having a stronger cross-attention modeling achieve a greater adversarial robustness on tracking datasets, such as VOT2022ST and GOT10k. Our results also indicate the necessity for new attack methods to effectively tackle the latest types of transformer trackers. The codes necessary to reproduce this study are available at <a class="link-external link-https" href="https://github.com/fatemehN/ReproducibilityStudy" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem this paper attempts to address is the evaluation of the robustness of Transformer-based object trackers under adversarial attacks. Specifically, the paper investigates the impact of different types of adversarial attacks on both Transformer-based and non-Transformer-based trackers through a series of experiments, and explores the performance of these attacks under different parameter settings (such as perturbation levels, noise bounds, etc.). Additionally, the paper pays special attention to the adversarial robustness based on bounding box prediction and binary mask prediction. ### Main Issues: 1. **Evaluating the effectiveness of existing adversarial attacks on Transformer-based trackers**: The paper extends adversarial attack methods originally designed for non-Transformer trackers to assess their effectiveness on Transformer-based trackers. 2. **Studying the performance of different attack methods at different perturbation levels**: The paper analyzes the performance of different attack methods at various perturbation levels and explores the impact of changes in perturbation levels on overall tracking results. 3. **Comparing the robustness of Transformer-based and non-Transformer-based trackers under adversarial attacks**: Through comparative experiments, the paper evaluates whether Transformer-based trackers are more robust to adversarial attacks than non-Transformer-based trackers. 4. **Exploring the performance of new Transformer trackers under adversarial attacks**: The paper investigates the robustness of new Transformer trackers with stronger cross-attention modeling under adversarial attacks, particularly on the VOT2022ST and GOT10k datasets. ### Research Background: - **Existing Research**: Although Transformer-based trackers have shown excellent performance in the latest benchmarks, their robustness under adversarial attacks has not been fully studied. - **Challenges**: Due to the structural differences between Transformer trackers and traditional multi-head pipelines, traditional adversarial attack methods may not be applicable to these new trackers. - **Research Significance**: Evaluating adversarial attacks on Transformer-based trackers can provide a better understanding of the weaknesses of these models and guide the development of more robust trackers. ### Experimental Setup: - **Datasets**: VOT2022ST, UAV123, DAVIS2016, and GOT10k. - **Trackers**: Including 3 Transformer-based trackers (TransT, MixFormer, ROMTrack) and 4 trackers based on other architectures (SiamRPN, DaSiamRPN, DiMP, PrDiMP). - **Attack Methods**: SPARK, RTAA, IoU, and CSA. ### Main Contributions: 1. **Extended the application scope of adversarial attacks**: Applied attack methods originally designed for non-Transformer trackers to Transformer-based trackers. 2. **Comprehensively evaluated the adversarial robustness of Transformer-based trackers**: Conducted evaluations under different output scenarios, perturbation levels, and noise bound changes. 3. **Revealed the robustness advantages of new Transformer trackers**: Particularly those with stronger cross-attention modeling, which showed higher robustness under adversarial attacks. Through these studies, the paper provides important references and guidance for improving the adversarial robustness of Transformer-based trackers.

Reproducibility Study on Adversarial Attacks Against Robust Transformer Trackers

Protego: Detecting Adversarial Examples for Vision Transformers Via Intrinsic Capabilities

Security in Transformer Visual Trackers: A Case Study on the Adversarial Robustness of Two Models

Adversarial Bounding Boxes Generation (ABBG) Attack against Visual Object Trackers

F&F Attack: Adversarial Attack Against Multiple Object Trackers by Inducing False Negatives and False Positives

The Efficacy of Transformer-based Adversarial Attacks in Security Domains

Pluggable Attack for Visual Object Tracking

TrackPGD: A White-box Attack using Binary Masks against Robust Transformer Trackers

Evaluating the Adversarial Robustness of Detection Transformers

Evaluating the Robustness of LiDAR Point Cloud Tracking Against Adversarial Attack

Bilateral Adversarial Patch Generating Network for the Object Tracking Algorithm.

On the Robustness of Vision Transformers to Adversarial Examples

Robust Deep Object Tracking against Adversarial Attacks

A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models

Efficient Adversarial Attacks for Visual Object Tracking

Robustness and Transferability of Adversarial Attacks on Different Image Classification Neural Networks

Propagating Prior Information with Transformer for Robust Visual Object Tracking

Robust Tracking against Adversarial Attacks

BankTweak: Adversarial Attack against Multi-Object Trackers by Manipulating Feature Banks

Robustness analysis of Visual Transformer based on adversarial attacks