Abstract:Deep neural network based systems deployed in sensitive environments are vulnerable to adversarial attacks. Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic. Recent works have utilized the diffusion inversion process to map images into a latent space, where high-level semantics are manipulated by introducing perturbations. However, they often results in substantial semantic distortions in the denoised output and suffers from low efficiency. In this study, we propose a novel framework called Semantic-Consistent Unrestricted Adversarial Attacks (SCA), which employs an inversion method to extract edit-friendly noise maps and utilizes Multimodal Large Language Model (MLLM) to provide semantic guidance throughout the process. Under the condition of rich semantic information provided by MLLM, we perform the DDPM denoising process of each step using a series of edit-friendly noise maps, and leverage DPM Solver++ to accelerate this process, enabling efficient sampling with semantic consistency. Compared to existing methods, our framework enables the efficient generation of adversarial examples that exhibit minimal discernible semantic changes. Consequently, we for the first time introduce Semantic-Consistent Adversarial Examples (SCAE). Extensive experiments and visualizations have demonstrated the high efficiency of SCA, particularly in being on average 12 times faster than the state-of-the-art attacks. Our research can further draw attention to the security of multimedia information.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the vulnerability of deep neural network (DNN) models to adversarial examples, especially the challenges faced in generating unrestricted adversarial examples with high semantic consistency. Specifically, existing methods often lead to obvious semantic distortion and are less efficient when generating adversarial examples. To overcome these problems, the authors propose a new framework - Semantic - Consistent Unrestricted Adversarial Attack (SCA), aiming to maintain the semantic consistency of images while improving the generation efficiency when generating adversarial examples. ### Main problems 1. **Semantic distortion**: When existing unrestricted adversarial attack methods generate adversarial examples, although they can deceive DNN models, they often introduce significant semantic changes, resulting in the generated adversarial examples being visually unnatural and easily detectable by humans. 2. **Low generation efficiency**: Traditional methods usually require a large number of time steps to accurately map images to the latent space, which makes the attack process time - consuming and prone to uncontrollable semantic changes. ### Solutions To solve the above problems, the SCA framework proposes the following innovations: 1. **Semantic Fixation Inversion**: - Through an improved inversion mechanism, the clean image is more strongly "imprinted" on the noise map, thereby retaining more semantic information in the latent space. - Use the Multimodal Large Language Model (MLLM) to generate detailed image descriptions as semantic guidance to ensure that the perturbation direction in the latent space is restricted, thus maintaining semantic consistency. 2. **Semantically Guided Perturbation**: - Optimize the adversarial objective in the latent space, use rich semantic priors to guide the perturbation process, so that the generated adversarial examples can deceive DNN models without significantly changing the overall semantics of the image. - Introduce DPM Solver++ to accelerate the sampling process, reduce the required number of time steps, and significantly improve the generation efficiency. ### Experimental results The experimental results show that the SCA framework is about 12 times faster than the existing state - of - the - art methods in generating adversarial examples while maintaining semantic consistency. In addition, the generated adversarial examples are more visually natural, difficult to be detected by humans, and also improve the success rate and transferability of the attack. Through these improvements, the SCA framework not only improves the concealment and efficiency of adversarial attacks, but also provides new insights into understanding the vulnerability of DNNs and provides a reference for developing new defense strategies.

SCA: Highly Efficient Semantic-Consistent Unrestricted Adversarial Attack

SCA: Highly Efficient Semantic-Consistent Unrestricted Adversarial Attack

CSFAdv: Critical Semantic Fusion Guided Least-Effort Adversarial Example Attacks

SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing

Stealthy Adversarial Examples for Semantic Segmentation in Remote Sensing

Content-based Unrestricted Adversarial Attack

Cascaded Adversarial Attack: Simultaneously Fooling Rain Removal and Semantic Segmentation Networks

Sensitive Region-Aware Black-Box Adversarial Attacks.

On the Robustness of Semantic Segmentation Models to Adversarial Attacks

Demiguise Attack: Crafting Invisible Semantic Adversarial Perturbations with Perceptual Similarity

To Make Yourself Invisible with Adversarial Semantic Contours

Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack

Semantic Image Attack for Visual Model Diagnosis

Attention-SA: Exploiting Model-approximated Data Semantics for Adversarial Attack

Enhancing robustness in video recognition models: Sparse adversarial attacks and beyond

SSTA: Salient Spatially Transformed Attack

Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models

SingleADV: Single-Class Target-Specific Attack Against Interpretable Deep Learning Systems

AdvDenoise: Fast Generation Framework of Universal and Robust Adversarial Patches Using Denoise

MMAD-Purify: A Precision-Optimized Framework for Efficient and Scalable Multi-Modal Attacks

Natural Language Induced Adversarial Images