Abstract:The data fusion systems between multiple modals have attracted a wide range of concerns about their adversarial robustness. Recent image captioning attacks lack cross-structure transferability against models with diverse structures. Therefore, the attackers demonstrate satisfactory performance only when they gain full or partial knowledge of target image captioning models. So far, the cross-structure transferability of multi-modal adversarial examples has not been thoroughly investigated. In this paper, a theorem is proposed to analyze the upper bound captioning adversarial transferability. We design a new transfer-based adversarial attack method (Cascade & Allocate) against image captioning models. Our method can be divided into two steps. The first step randomly selects a set of candidate models with diverse structures, and we use a momentum-like strategy to generate perturbations. This ‘model cascade’ step narrows the gap of the gradient directions between different image captioning models, therefore enhancing the cross-model transferability and generating model-agnostic adversarial perturbations. The second step utilizes the upper bound transferability theorem to allocate the weight of perturbations per model. This ‘weight allocate’ step enhances the weight of model with the most consistent gradient in all candidate models, which generates more transferable adversarial examples. In the experiments, we compare our approach with other captioning and ensemble-based attacks against five black-box models with different structures on MS COCO and Flickr-30k datasets. The adversarial examples exhibit state-of-the-art adversarial transferability against five black-box models with different structures. The results demonstrate that our approach is practical and generalizes well to a wide range of captioning models.

A [72, 36, 16] doubly even code does not have an automorphism of order 11

The Central Limit Theorem for the Normalized Sums of the MAI for SSMA Communication Systems Using Spreading Sequences of Markov Chains

Boosting the Transferability of Video Adversarial Examples Via Temporal Translation.

Transferable Adversarial Attacks for Image and Video Object Detection

SS-CMT: a label independent cross-modal transferable adversarial video attack with sparse strategy

Enhancing Transferability of Adversarial Examples Through Mixed-Frequency Inputs

Transferable Adversarial Attack for Both Vision Transformers and Convolutional Networks Via Momentum Integrated Gradients

An Optimized Transfer Attack Framework Towards Multi-Modal Machine Learning

Admix: Enhancing the Transferability of Adversarial Attacks

Improving Transferability of Adversarial Examples With Input Diversity

Heuristic Black-box Adversarial Attacks on Video Recognition Models

Towards transferable adversarial attacks on vision transformers for image classification

Black-box Adversarial Attacks on Video Recognition Models

Bag of Tricks to Boost Adversarial Transferability

Delving into Transferable Adversarial Examples and Black-box Attacks

Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks

Improving Adversarial Transferability of Vision-Language Pre-training Models through Collaborative Multimodal Interaction

Enhancing the Adversarial Transferability with Channel Decomposition

Cascade & allocate: A cross-structure adversarial attack against models fusing vision and language

Dual stage black-box adversarial attack against vision transformer

Attention‐guided transformation‐invariant attack for black‐box adversarial examples