Abstract:Advances in deep learning have led to the broad adoption of Deep Neural Networks (DNNs) to a range of important machine learning problems, e.g., guiding autonomous vehicles, speech recognition, malware detection. Yet, machine learning models, including DNNs, were shown to be vulnerable to adversarial samples-subtly (and often humanly indistinguishably) modified malicious inputs crafted to compromise the integrity of their outputs. Adversarial examples thus enable adversaries to manipulate system behaviors. Potential attacks include attempts to control the behavior of vehicles, have spam content identified as legitimate content, or have malware identified as legitimate software. Adversarial examples are known to transfer from one model to another, even if the second model has a different architecture or was trained on a different set. We introduce the first practical demonstration that this cross-model transfer phenomenon enables attackers to control a remotely hosted DNN with no access to the model, its parameters, or its training data. In our demonstration, we only assume that the adversary can observe outputs from the target DNN given inputs chosen by the adversary. We introduce the attack strategy of fitting a substitute model to the input-output pairs in this manner, then crafting adversarial examples based on this auxiliary model. We evaluate the approach on existing DNN datasets and real-world settings. In one experiment, we force a DNN supported by MetaMind (one of the online APIs for DNN classifiers) to mis-classify inputs at a rate of 84.24%. We conclude with experiments exploring why adversarial samples transfer between DNNs, and a discussion on the applicability of our attack when targeting machine learning algorithms distinct from DNNs.

Adversarial Examples: Attacks and Defenses for Deep Learning

Adversarial Example Attacks in Internet of Things (IoT)

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Adversarial Examples in Deep Learning: Characterization and Divergence

How Can We Deal with Adversarial Examples?

Adversarial examples: attacks and defenses in the physical world

Interpreting Adversarial Examples in Deep Learning: A Review

Adversarial Examples: Opportunities and Challenges

Adversarial Attacks and Defenses in Deep Learning: From a Perspective of Cybersecurity

Adversarial Attack? Don't Panic

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

A Survey of Defense Methods Against Adversarial Examples

Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

Adversarial Attacks and Defenses Against Deep Neural Networks: A Survey

The Vulnerability of the Neural Networks Against Adversarial Examples in Deep Learning Algorithms

Adversarial Examples in Physical World

Research Progress and Challenges on Application-Driven Adversarial Examples: A Survey

Adversarial examples: attacks and defences on medical deep learning systems

Adversarial Examples Against the Deep Learning Based Network Intrusion Detection Systems

Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples