Ml-loo: Detecting adversarial examples with feature attribution

Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael Jordan

2020-04-03

Abstract:

What problem does this paper attempt to address?

An Adversarial Attack Via Feature Contributive Regions

Yaguan Qian,Jiamin Wang,Xiang Ling,Zhaoquan Gu,Bin Wang,Chunming Wu

2021-01-01

Abstract:Recently, to deal with the vulnerability to generate examples of CNNs, there are many advanced algorithms that have been proposed. These algorithms focus on modifying global pixels directly with small perturbations, and some work involves modifying local pixels. However, the global attacks have the problem of perturbations’ redundancy and the local attacks are not effective. To overcome this challenge, we achieve a trade-off between the perturbation power and the number of perturbed pixels in this paper. The key idea is to find the feature contributive regions (FCRs) of the images. Furthermore, in order to create an adversarial example similar to the corresponding clean image as much as possible, we redefine a loss function as the objective function of the optimization in this paper and then using gradient descent optimization algorithm to find the efficient perturbations. Various experiments have been carried out on CIFAR-10 and ILSVRC2012 datasets, which show the excellence of this method, and in addition, the FCRs attack shows strong attack ability in both white-box and black-box settings.
Attribution-driven Causal Analysis for Detection of Adversarial Examples

Susmit Jha,Sunny Raj,Steven Lawrence Fernandes,Sumit Kumar Jha,Somesh Jha,Gunjan Verma,Brian Jalaian,Ananthram Swami

DOI: https://doi.org/10.48550/arXiv.1903.05821

2019-03-14

Abstract:Attribution methods have been developed to explain the decision of a machine learning model on a given input. We use the Integrated Gradient method for finding attributions to define the causal neighborhood of an input by incrementally masking high attribution features. We study the robustness of machine learning models on benign and adversarial inputs in this neighborhood. Our study indicates that benign inputs are robust to the masking of high attribution features but adversarial inputs generated by the state-of-the-art adversarial attack methods such as DeepFool, FGSM, CW and PGD, are not robust to such masking. Further, our study demonstrates that this concentration of high-attribution features responsible for the incorrect decision is more pronounced in physically realizable adversarial examples. This difference in attribution of benign and adversarial inputs can be used to detect adversarial examples. Such a defense approach is independent of training data and attack method, and we demonstrate its effectiveness on digital and physically realizable perturbations.

Machine Learning
Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks

Marissa Dotter,Sherry Xie,Keith Manville,Josh Harguess,Colin Busho,Mikel Rodriguez

DOI: https://doi.org/10.48550/arXiv.2101.02899

IF: 5.414

2021-01-08

Machine Learning

Abstract:Machine Learning (ML) models are known to be vulnerable to adversarial inputs and researchers have demonstrated that even production systems, such as self-driving cars and ML-as-a-service offerings, are susceptible. These systems represent a target for bad actors. Their disruption can cause real physical and economic harm. When attacks on production ML systems occur, the ability to attribute the attack to the responsible threat group is a critical step in formulating a response and holding the attackers accountable. We pose the following question: can adversarially perturbed inputs be attributed to the particular methods used to generate the attack? In other words, is there a way to find a signal in these attacks that exposes the attack algorithm, model architecture, or hyperparameters used in the attack? We introduce the concept of adversarial attack attribution and create a simple supervised learning experimental framework to examine the feasibility of discovering attributable signals in adversarial attacks. We find that it is possible to differentiate attacks generated with different attack algorithms, models, and hyperparameters on both the CIFAR-10 and MNIST datasets.
A Novel Adversarial Detection Method for UAV Vision Systems Via Attribution Maps

Zhun Zhang,Qihe Liu,Chunjiang Wu,Shijie Zhou,Zhangbao Yan

DOI: https://doi.org/10.3390/drones7120697

IF: 5.532

2023-01-01

Drones

Abstract:With the rapid advancement of unmanned aerial vehicles (UAVs) and the Internet of Things (IoTs), UAV-assisted IoTs has become integral in areas such as wildlife monitoring, disaster surveillance, and search and rescue operations. However, recent studies have shown that these systems are vulnerable to adversarial example attacks during data collection and transmission. These attacks subtly alter input data to trick UAV-based deep learning vision systems, significantly compromising the reliability and security of IoTs systems. Consequently, various methods have been developed to identify adversarial examples within model inputs, but they often lack accuracy against complex attacks like C&W and others. Drawing inspiration from model visualization technology, we observed that adversarial perturbations markedly alter the attribution maps of clean examples. This paper introduces a new, effective detection method for UAV vision systems that uses attribution maps created by model visualization techniques. The method differentiates between genuine and adversarial examples by extracting their unique attribution maps and then training a classifier on these maps. Validation experiments on the ImageNet dataset showed that our method achieves an average detection accuracy of 99.58%, surpassing the state-of-the-art methods.
Adversarial Examples Detection with Enhanced Image Difference Features based on Local Histogram Equalization

Zhaoxia Yin,Shaowei Zhu,Hang Su,Jianteng Peng,Wanli Lyu,Bin Luo

DOI: https://doi.org/10.48550/arxiv.2305.04436

2023-01-01

Abstract: Deep Neural Networks (DNNs) have recently made significant progress in many fields. However, studies have shown that DNNs are vulnerable to adversarial examples, where imperceptible perturbations can greatly mislead DNNs even if the full underlying model parameters are not accessible. Various defense methods have been proposed, such as feature compression and gradient masking. However, numerous studies have proven that previous methods create detection or defense against certain attacks, which renders the method ineffective in the face of the latest unknown attack methods. The invisibility of adversarial perturbations is one of the evaluation indicators for adversarial example attacks, which also means that the difference in the local correlation of high-frequency information in adversarial examples and normal examples can be used as an effective feature to distinguish the two. Therefore, we propose an adversarial example detection framework based on a high-frequency information enhancement strategy, which can effectively extract and amplify the feature differences between adversarial examples and normal examples. Experimental results show that the feature augmentation module can be combined with existing detection models in a modular way under this framework. Improve the detector's performance and reduce the deployment cost without modifying the existing detection model.
Generating Adversarial Examples Against Remote Sensing Scene Classification Via Feature Approximation

Rui Zhu,Shiping Ma,Jiawei Lian,Linyuan He,Shaohui Mei

DOI: https://doi.org/10.1109/jstars.2024.3399780

IF: 4.715

2024-01-01

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Abstract:The existence of adversarial examples highlights the vulnerability of deep neural networks, which can change the recognition results by adding well-designed perturbations to the original image. It brings a great challenge to the remote sensing images (RSI) scene classification. RSI scene classification primarily relies on the spatial and texture feature information of images, making attacks in the feature domain more effective. In this study, we introduce the feature approximation (FA) strategy, which generates adversarial examples by approximating clean image features to virtual images that are designed to not belong to any category. Our research aims to attack image classification models that are trained with RSI and discover the common vulnerabilities of these models. Specifically, we benchmark the FA attack using both featureless images and images generated via data augmentation methods. We then extend the FA attack to multimodel FA (MFA), improving the transferability of the attack. Finally, we show that the FA strategy is also effective for targeted attacks by approximating the input clean image features to the target category image features. Extensive experiments on the remote sensing classification datasets UC Merced and AID demonstrate the effectiveness of the methods in this article. The FA attack exhibits remarkable attack performance. Furthermore, the proposed MFA attack outperforms the success rate achieved by existing advanced targetless black-box attacks by an average of over 15%. The FA attack also performs better compared to multiple existing targeted white-box attacks.
ARIA: Adversarially Robust Image Attribution for Content Provenance

Maksym Andriushchenko,Xiaoyang Rebecca Li,Geoffrey Oxholm,Thomas Gittings,Tu Bui,Nicolas Flammarion,John Collomosse

DOI: https://doi.org/10.48550/arXiv.2202.12860

2022-02-26

Abstract:Image attribution -- matching an image back to a trusted source -- is an emerging tool in the fight against online misinformation. Deep visual fingerprinting models have recently been explored for this purpose. However, they are not robust to tiny input perturbations known as adversarial examples. First we illustrate how to generate valid adversarial images that can easily cause incorrect image attribution. Then we describe an approach to prevent imperceptible adversarial attacks on deep visual fingerprinting models, via robust contrastive learning. The proposed training procedure leverages training on $\ell_\infty$-bounded adversarial examples, it is conceptually simple and incurs only a small computational overhead. The resulting models are substantially more robust, are accurate even on unperturbed images, and perform well even over a database with millions of images. In particular, we achieve 91.6% standard and 85.1% adversarial recall under $\ell_\infty$-bounded perturbations on manipulated images compared to 80.1% and 0.0% from prior work. We also show that robustness generalizes to other types of imperceptible perturbations unseen during training. Finally, we show how to train an adversarially robust image comparator model for detecting editorial changes in matched images.

Computer Vision and Pattern Recognition,Machine Learning
On the (Statistical) Detection of Adversarial Examples

Kathrin Grosse,Praveen Manoharan,Nicolas Papernot,Michael Backes,Patrick McDaniel

DOI: https://doi.org/10.48550/arXiv.1702.06280

2017-02-21

Cryptography and Security

Abstract:Machine Learning (ML) models are applied in a variety of tasks such as network intrusion detection or Malware classification. Yet, these models are vulnerable to a class of malicious inputs known as adversarial examples. These are slightly perturbed inputs that are classified incorrectly by the ML model. The mitigation of these adversarial inputs remains an open problem. As a step towards understanding adversarial examples, we show that they are not drawn from the same distribution than the original data, and can thus be detected using statistical tests. Using thus knowledge, we introduce a complimentary approach to identify specific inputs that are adversarial. Specifically, we augment our ML model with an additional output, in which the model is trained to classify all adversarial inputs. We evaluate our approach on multiple adversarial example crafting methods (including the fast gradient sign and saliency map methods) with several datasets. The statistical test flags sample sets containing adversarial inputs confidently at sample sizes between 10 and 100 data points. Furthermore, our augmented model either detects adversarial examples as outliers with high accuracy (> 80%) or increases the adversary's cost - the perturbation added - by more than 150%. In this way, we show that statistical properties of adversarial examples are essential to their detection.
Detecting Adversarial Image Examples in Deep Neural Networks with Adaptive Noise Reduction

Bin Liang,Hongcheng Li,Miaoqiang Su,Xirong Li,Wenchang Shi,Xiaofeng Wang

DOI: https://doi.org/10.1109/tdsc.2018.2874243

2018-01-01

IEEE Transactions on Dependable and Secure Computing

Abstract:Recently, many studies have demonstrated deep neural network (DNN) classifiers can be fooled by the adversarial example, which is crafted via introducing some perturbations into an original sample. Accordingly, some powerful defense techniques were proposed. However, existing defense techniques often require modifying the target model or depend on the prior knowledge of attacks. In this paper, we propose a straightforward method for detecting adversarial image examples, which can be directly deployed into unmodified off-the-shelf DNN models. We consider the perturbation to images as a kind of noise and introduce two classic image processing techniques, scalar quantization and smoothing spatial filter, to reduce its effect. The image entropy is employed as a metric to implement an adaptive noise reduction for different kinds of images. Consequently, the adversarial example can be effectively detected by comparing the classification results of a given sample and its denoised version, without referring to any prior knowledge of attacks. More than 20,000 adversarial examples against some state-of-the-art DNN models are used to evaluate the proposed method, which are crafted with different attack techniques. The experiments show that our detection method can achieve a high overall F1 score of 96.39 percent and certainly raises the bar for defense-aware attacks.
Adversarial Examples in Deep Learning: Characterization and Divergence

Wenqi Wei,Ling Liu,Margaret Loper,Stacey Truex,Lei Yu,Mehmet Emre Gursoy,Yanzhao Wu

DOI: https://doi.org/10.48550/arXiv.1807.00051

IF: 5.414

2018-06-29

Machine Learning

Abstract:The burgeoning success of deep learning has raised the security and privacy concerns as more and more tasks are accompanied with sensitive data. Adversarial attacks in deep learning have emerged as one of the dominating security threat to a range of mission-critical deep learning systems and applications. This paper takes a holistic and principled approach to perform statistical characterization of adversarial examples in deep learning. We provide a general formulation of adversarial examples and elaborate on the basic principle for adversarial attack algorithm design. We introduce easy and hard categorization of adversarial attacks to analyze the effectiveness of adversarial examples in terms of attack success rate, degree of change in adversarial perturbation, average entropy of prediction qualities, and fraction of adversarial examples that lead to successful attacks. We conduct extensive experimental study on adversarial behavior in easy and hard attacks under deep learning models with different hyperparameters and different deep learning frameworks. We show that the same adversarial attack behaves differently under different hyperparameters and across different frameworks due to the different features learned under different deep learning model training process. Our statistical characterization with strong empirical evidence provides a transformative enlightenment on mitigation strategies towards effective countermeasures against present and future adversarial attacks.
Feature-Filter: Detecting Adversarial Examples through Filtering off Recessive Features

Hui Liu,Bo Zhao,Minzhi Ji,Yuefeng Peng,Jiabao Guo,Peng Liu

DOI: https://doi.org/10.48550/arXiv.2107.09502

2021-11-29

Abstract:Deep neural networks (DNNs) are under threat from adversarial example attacks. The adversary can easily change the outputs of DNNs by adding small well-designed perturbations to inputs. Adversarial example detection is a fundamental work for robust DNNs-based service. Adversarial examples show the difference between humans and DNNs in image recognition. From a human-centric perspective, image features could be divided into dominant features that are comprehensible to humans, and recessive features that are incomprehensible to humans, yet are exploited by DNNs. In this paper, we reveal that imperceptible adversarial examples are the product of recessive features misleading neural networks, and an adversarial attack is essentially a kind of method to enrich these recessive features in the image. The imperceptibility of the adversarial examples indicates that the perturbations enrich recessive features, yet hardly affect dominant features. Therefore, adversarial examples are sensitive to filtering off recessive features, while benign examples are immune to such operation. Inspired by this idea, we propose a label-only adversarial detection approach that is referred to as feature-filter. Feature-filter utilizes discrete cosine transform to approximately separate recessive features from dominant features, and gets a mutant image that is filtered off recessive features. By only comparing DNN's prediction labels on the input and its mutant, feature-filter can real-time detect imperceptible adversarial examples at high accuracy and few false positives.

Machine Learning
Model-agnostic Adversarial Example Detection via High-Frequency Amplification

Qiao Li,Jing Chen,Kun He,Zijun Zhang,Ruiying Du,Jisi She,Xinxin Wang

DOI: https://doi.org/10.1016/j.cose.2024.103791

IF: 5.105

2024-02-01

Computers & Security

Abstract:Image classification based on Deep Neural Networks (DNNs) is vulnerable to adversarial examples, which make the classifier output incorrect predictions. One approach to defending against this attack is to detect whether the input is an adversarial example. Unfortunately, existing adversarial example detection methods heavily rely on the underlying classifier and may fail when the classifier is upgraded. In this paper, we propose a model-agnostic detection method that leverages high-frequency signals from adversarial noises in adversarial examples and does not need interactions with the underlying classifier. We amplify redundant high-frequency signals brought by adversarial noises and represent object boundaries with these signals in an image. Our key insight is that the boundaries extracted by redundant high-frequency signals have a strong correlation with the boundaries of images in adversarial examples, while this correlation does not exist in clean images. Furthermore, adversarial examples of large images have more high-frequency signals and make adversarial detection easier on large image datasets. Experimental results show that our method has good transferability and can accurately detect various adversarial examples on different datasets.

computer science, information systems
LSD: Adversarial Examples Detection Based on Label Sequences Discrepancy

Shigeng Zhang,Shuxin Chen,Chengyao Hua,Zhetao Li,Yanchun Li,Xuan Liu,Kai Chen,Zhankai Li,Weiping Wang

DOI: https://doi.org/10.1109/tifs.2023.3304455

IF: 7.231

2023-08-25

IEEE Transactions on Information Forensics and Security

Abstract:Deep neural network (DNN) models have been widely used in many tasks due to their superior performance. However, DNN models are usually vulnerable to adversarial example attacks, which limits their applications in many safety-critic scenarios. How to effectively detect adversarial examples to enhance the robustness of DNN models has attracted much attention in recent years. Most adversarial example detection methods require modifying or retraining the model, which is impractical and reduces the classification accuracy of normal examples. In this paper, we propose an adversarial example detection approach that does not require modification of the DNN models and meanwhile retains the classification accuracy of normal examples. The key observation is that when we transform the input example with some operations (e.g., masking a pixel with a reference value), feed the transformed example to the target model, and use the output of the intermediate layers to predict the label of the example, the generated label sequences of adversarial examples will be extremely discrepant but the label sequences of normal examples keep nearly unchanged. Motivated by this observation, we design an approach to detect adversarial examples based on the label sequence discrepancy (LSD) of the given examples. The experimental results against five mainstream adversarial attacks on three benchmark datasets demonstrate that LSD outperforms the state-of-the-art solutions in the detection rate of adversarial examples. Moreover, LSD performs well at various confidence levels and exhibits good generalizability between different attacks.

computer science, theory & methods,engineering, electrical & electronic
Adversarial Examples Are Not Real Features

Ang Li,Yifei Wang,Yiwen Guo,Yisen Wang

2024-05-06

Abstract:The existence of adversarial examples has been a mystery for years and attracted much interest. A well-known theory by \citet{ilyas2019adversarial} explains adversarial vulnerability from a data perspective by showing that one can extract non-robust features from adversarial examples and these features alone are useful for classification. However, the explanation remains quite counter-intuitive since non-robust features are mostly noise features to humans. In this paper, we re-examine the theory from a larger context by incorporating multiple learning paradigms. Notably, we find that contrary to their good usefulness under supervised learning, non-robust features attain poor usefulness when transferred to other self-supervised learning paradigms, such as contrastive learning, masked image modeling, and diffusion models. It reveals that non-robust features are not really as useful as robust or natural features that enjoy good transferability between these paradigms. Meanwhile, for robustness, we also show that naturally trained encoders from robust features are largely non-robust under AutoAttack. Our cross-paradigm examination suggests that the non-robust features are not really useful but more like paradigm-wise shortcuts, and robust features alone might be insufficient to attain reliable model robustness. Code is available at \url{<a class="link-external link-https" href="https://github.com/PKU-ML/AdvNotRealFeatures" rel="external noopener nofollow">this https URL</a>}.

Machine Learning,Computer Vision and Pattern Recognition
Simple Transparent Adversarial Examples

Jaydeep Borkar,Pin-Yu Chen

DOI: https://doi.org/10.48550/arXiv.2105.09685

2021-05-20

Abstract:There has been a rise in the use of Machine Learning as a Service (MLaaS) Vision APIs as they offer multiple services including pre-built models and algorithms, which otherwise take a huge amount of resources if built from scratch. As these APIs get deployed for high-stakes applications, it's very important that they are robust to different manipulations. Recent works have only focused on typical adversarial attacks when evaluating the robustness of vision APIs. We propose two new aspects of adversarial image generation methods and evaluate them on the robustness of Google Cloud Vision API's optical character recognition service and object detection APIs deployed in real-world settings such as <a class="link-external link-http" href="http://sightengine.com" rel="external noopener nofollow">this http URL</a>, <a class="link-external link-http" href="http://picpurify.com" rel="external noopener nofollow">this http URL</a>, Google Cloud Vision API, and Microsoft Azure's Computer Vision API. Specifically, we go beyond the conventional small-noise adversarial attacks and introduce secret embedding and transparent adversarial examples as a simpler way to evaluate robustness. These methods are so straightforward that even non-specialists can craft such attacks. As a result, they pose a serious threat where APIs are used for high-stakes applications. Our transparent adversarial examples successfully evade state-of-the art object detections APIs such as Azure Cloud Vision (attack success rate 52%) and Google Cloud Vision (attack success rate 36%). 90% of the images have a secret embedded text that successfully fools the vision of time-limited humans but is detected by Google Cloud Vision API's optical character recognition. Complementing to current research, our results provide simple but unconventional methods on robustness evaluation.

Computer Vision and Pattern Recognition,Artificial Intelligence,Cryptography and Security,Machine Learning
Adversarial Analysis for Source Camera Identification

Bo Wang,Mengnan Zhao,Wei Wang,Xiaorui Dai,Yi Li,Yanqing Guo

DOI: https://doi.org/10.1109/tcsvt.2020.3047084

IF: 5.859

2021-11-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Recent studies highlight the vulnerability of convolutional neural networks (CNNs) to adversarial attacks, which also calls into question the reliability of forensic methods. Existing adversarial attacks generate one-to-one noise, which means these methods have not learned the fingerprint information. Therefore, we introduce two powerful attacks, fingerprint copy-move attack, and joint feature-based auto-learning attack. To validate the performance of attack methods, we move a step ahead and introduce the higher possible defense mechanism relation mismatch. which expands the characterization differences of classifiers in the same classification network. Extensive experiments show that relation mismatch is superior in recognizing adversarial examples and prove that the proposed fingerprint-based attacks are more powerful. Both proposed attacks show excellent attack transferability to unknown samples. The Pytorch® implementations of these methods can download from an open-source GitHub project https://github.com/Dlut-lab-zmn/Source-attack.

engineering, electrical & electronic
Detecting Adversarial Examples by Additional Evidence from Noise Domain

Song Gao,Shui Yu,Liwen Wu,Shaowen Yao,Xiaowei Zhou

DOI: https://doi.org/10.1049/ipr2.12354

IF: 2.3

2021-01-01

IET Image Processing

Abstract:Deep neural networks are widely adopted powerful tools for perceptual tasks. However, recent research indicated that they are easily fooled by adversarial examples, which are produced by adding imperceptible adversarial perturbations to clean examples. Here the steganalysis rich model (SRM) is utilized to generate noise feature maps, and they are combined with RGB images to discover the difference between adversarial examples and clean examples. In particular, a two-stream pseudo-siamese network that fuses the subtle difference in RGB images with the noise inconsistency in noise features is proposed. The proposed method has strong detection capability and transferability, and can be combined with any model without modifying its architecture or training procedure. The extensive empirical experiments show that, compared with the state-of-the-art detection methods, the proposed approach achieves excellent performance in distinguishing adversarial samples generated by popular attack methods on different real datasets. Moreover, this method has good generalization, it trained by a specific adversary can defend against other adversaries effectively.
Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples

Chumeng Liang,Xiaoyu Wu,Yang Hua,Jiaru Zhang,Yiming Xue,Tao Song,Zhengui Xue,Ruhui Ma,Haibing Guan

2023-06-06

Abstract:Recently, Diffusion Models (DMs) boost a wave in AI for Art yet raise new copyright concerns, where infringers benefit from using unauthorized paintings to train DMs to generate novel paintings in a similar style. To address these emerging copyright violations, in this paper, we are the first to explore and propose to utilize adversarial examples for DMs to protect human-created artworks. Specifically, we first build a theoretical framework to define and evaluate the adversarial examples for DMs. Then, based on this framework, we design a novel algorithm, named AdvDM, which exploits a Monte-Carlo estimation of adversarial examples for DMs by optimizing upon different latent variables sampled from the reverse process of DMs. Extensive experiments show that the generated adversarial examples can effectively hinder DMs from extracting their features. Therefore, our method can be a powerful tool for human artists to protect their copyright against infringers equipped with DM-based AI-for-Art applications. The code of our method is available on GitHub: <a class="link-external link-https" href="https://github.com/mist-project/mist.git" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence,Cryptography and Security,Machine Learning
PIP: Detecting Adversarial Examples in Large Vision-Language Models Via Attention Patterns of Irrelevant Probe Questions

Yudong Zhang,Ruobing Xie,Jiansheng Chen,Xingwu Sun,Yu Wang

DOI: https://doi.org/10.1145/3664647.3685510

2024-01-01

Abstract:Large Vision-Language Models (LVLMs) have demonstrated their powerful multimodal capabilities. However, they also face serious safety problems, as adversaries can induce robustness issues in LVLMs through the use of well-designed adversarial examples. Therefore, LVLMs are in urgent need of detection tools for adversarial examples to prevent incorrect responses. In this work, we first discover that LVLMs exhibit regular attention patterns for clean images when presented with probe questions. We propose an unconventional method named PIP, which utilizes the attention patterns of one randomly selected irrelevant probe question (e.g., "Is there a clock?") to distinguish adversarial examples from clean examples. Regardless of the image to be tested and its corresponding question, PIP only needs to perform one additional inference of the image to be tested and the probe question, and then achieves successful detection of adversarial examples. Even under black-box attacks and open dataset scenarios, our PIP, coupled with a simple SVM, still achieves more than 98 detect adversarial attacks on LVLMs via simple irrelevant probe questions, shedding light on deeper understanding and introspection within LVLMs. The code is available at https://github.com/btzyd/pip.
Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples

Guanhong Tao,Shiqing Ma,Yingqi Liu,Xiangyu Zhang

DOI: https://doi.org/10.48550/arXiv.1810.11580

2018-10-27

Abstract:Adversarial sample attacks perturb benign inputs to induce DNN misbehaviors. Recent research has demonstrated the widespread presence and the devastating consequences of such attacks. Existing defense techniques either assume prior knowledge of specific attacks or may not work well on complex models due to their underlying assumptions. We argue that adversarial sample attacks are deeply entangled with interpretability of DNN models: while classification results on benign inputs can be reasoned based on the human perceptible features/attributes, results on adversarial samples can hardly be explained. Therefore, we propose a novel adversarial sample detection technique for face recognition models, based on interpretability. It features a novel bi-directional correspondence inference between attributes and internal neurons to identify neurons critical for individual attributes. The activation values of critical neurons are enhanced to amplify the reasoning part of the computation and the values of other neurons are weakened to suppress the uninterpretable part. The classification results after such transformation are compared with those of the original model to detect adversaries. Results show that our technique can achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false positives on benign inputs. In contrast, a state-of-the-art feature squeezing technique can only achieve 55% accuracy with 23.3% false positives.

Machine Learning,Artificial Intelligence,Cryptography and Security

Ml-loo: Detecting adversarial examples with feature attribution

An Adversarial Attack Via Feature Contributive Regions

Attribution-driven Causal Analysis for Detection of Adversarial Examples

Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks

A Novel Adversarial Detection Method for UAV Vision Systems Via Attribution Maps

Adversarial Examples Detection with Enhanced Image Difference Features based on Local Histogram Equalization

Generating Adversarial Examples Against Remote Sensing Scene Classification Via Feature Approximation

ARIA: Adversarially Robust Image Attribution for Content Provenance

On the (Statistical) Detection of Adversarial Examples

Detecting Adversarial Image Examples in Deep Neural Networks with Adaptive Noise Reduction

Adversarial Examples in Deep Learning: Characterization and Divergence

Feature-Filter: Detecting Adversarial Examples through Filtering off Recessive Features

Model-agnostic Adversarial Example Detection via High-Frequency Amplification

LSD: Adversarial Examples Detection Based on Label Sequences Discrepancy

Adversarial Examples Are Not Real Features

Simple Transparent Adversarial Examples

Adversarial Analysis for Source Camera Identification

Detecting Adversarial Examples by Additional Evidence from Noise Domain

Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples

PIP: Detecting Adversarial Examples in Large Vision-Language Models Via Attention Patterns of Irrelevant Probe Questions

Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples