Abstract:Deep learning models are vulnerable to attacks by adversarial examples. However, current studies are mainly limited to generating adversarial examples for specific models, and the migration of adversarial examples between different models is rarely studied. At the same time, in only studies, it is not considered that adding disturbance to the position of the image can improve the migration of adversarial examples better. As the main part of the picture, the model should give more weight to the foreground information in the recognition. Will adding more perturbations to the foreground information of the image result in a higher transfer attack rate? This paper focuses on the above problems, and proposes the FAPA algorithm, which first selects the foreground information of the image through the DINO framework, then uses the foreground information to generate M, and then uses PNA to generate the perturbation required for the whole picture. In order to show that our method attaches importance to the foreground information, we give a greater weight to the perturbation corresponding to the foreground information, and a smaller weight to the rest of the image. Finally, we optimize the generated perturbation through the gradient generated by the dual attack framework. In order to demonstrate the effectiveness of our method, we have conducted relevant comparative experiments. During the experiment, we used the three white-box ViTs models to attack the six black-box ViTs models and the three black-box CNNs models. In the transferable attack of ViTs models, the average attack success rate of our algorithm reaches 64.19%, which is much higher than 21.12% of the FGSM algorithm. In the transferable attack of CNN models, the average attack success rate of our algorithm reaches 48.07%, which is also higher than 18.65% of the FGSM algorithm. By integrating ViTs and CNNs models, the attack success rate of transfer of our algorithm reaches 56.13%, which is higher than 1.18% of the dual attack framework we refer to.

Towards transferable adversarial attacks on vision transformers for image classification

Protego: Detecting Adversarial Examples for Vision Transformers Via Intrinsic Capabilities

Towards Transferable Adversarial Attacks on Image and Video Transformers

Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers

Generating Transferable Adversarial Examples Against Vision Transformers

Dual stage black-box adversarial attack against vision transformer

Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization

Improving transferable adversarial attack for vision transformers via global attention and local drop

On Improving Adversarial Transferability of Vision Transformers

Attacking Transformers with Feature Diversity Adversarial Perturbation

Transferable Adversarial Attack for Both Vision Transformers and Convolutional Networks Via Momentum Integrated Gradients

TIA: Token Importance Transferable Attack on Vision Transformers.

Improving the Transferability of Adversarial Examples with Restructure Embedded Patches

Towards Efficient Adversarial Training on Vision Transformers

Query-Efficient Hard-Label Black-Box Attack against Vision Transformers

FAPA: Transferable Adversarial Attacks Based on Foreground Attention

Learning to Learn Transferable Attack

Exploring Transferable and Robust Adversarial Perturbation Generation Across Network Hierarchy

Defending Backdoor Attacks on Vision Transformer via Patch Processing

Enhancing Adversarial Transferability with Local Transformation

Enhancing the robustness of vision transformer defense against adversarial attacks based on squeeze-and-excitation module