Abstract:Prior studies have unveiled the vulnerability of the deep neural networks in the context of adversarial machine learning, leading to great recent attention into this area. One interesting question that has yet to be fully explored is the bias-variance relationship of adversarial machine learning, which can potentially provide deeper insights into this behaviour. The notion of bias and variance is one of the main approaches to analyze and evaluate the generalization and reliability of a machine learning model. Although it has been extensively used in other machine learning models, it is not well explored in the field of deep learning and it is even less explored in the area of adversarial machine learning. In this study, we investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network and analyze how adversarial perturbations can affect the generalization of a network. We derive the bias-variance trade-off for both classification and regression applications based on two main loss functions: (i) mean squared error (MSE), and (ii) cross-entropy. Furthermore, we perform quantitative analysis with both simulated and real data to empirically evaluate consistency with the derived bias-variance tradeoffs. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation from a bias-variance point of view and how this type of perturbation would change the performance of a network. Moreover, given these new theoretical findings, we introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies (e.g., PGD) while providing a high success rate in fooling deep neural networks in lower perturbation magnitudes.

Game Theoretical Adversarial Deep Learning with Variational Adversaries

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Deep Adversarial Learning for NLP.

Achieve Optimal Adversarial Accuracy for Adversarial Deep Learning using Stackelberg Game

Adversarial Example Games

A Game Theoretic Perspective on Adversarial Machine Learning and Related Cybersecurity Applications

A Direct Approach to Robust Deep Learning Using Adversarial Networks

Adversarial Examples: Attacks and Defenses for Deep Learning

Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples

Adversarial Examples in Deep Learning: Characterization and Divergence

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Vulnerability Under Adversarial Machine Learning: Bias or Variance?

Intriguing Properties of Adversarial Examples

Game-Theoretic Design of Secure and Resilient Distributed Support Vector Machines with Adversaries

Deviations in Representations Induced by Adversarial Attacks

Global Adversarial Attacks for Assessing Deep Learning Robustness

Exploring Adversarial Attacks on Neural Networks: An Explainable Approach

Defense against adversarial attacks on deep convolutional neural networks through nonlocal denoising

A Review of Adversarial Attacks in Computer Vision

An efficient adversarial example generation algorithm based on an accelerated gradient iterative fast gradient

Adversarial Examples on Object Recognition: A Comprehensive Survey