Abstract:Deep learning (DL) models are inherently vulnerable to adversarial examples - maliciously crafted inputs to trigger target DL models to misbehave - which significantly hinders the application of DL in security-sensitive domains. Intensive research on adversarial learning has led to an arms race between adversaries and defenders. Such plethora of emerging attacks and defenses raise many questions: Which attacks are more evasive, preprocessing-proof, or transferable? Which defenses are more effective, utility-preserving, or general? Are ensembles of multiple defenses more robust than individuals? Yet, due to the lack of platforms for comprehensive evaluation on adversarial attacks and defenses, these critical questions remain largely unsolved. In this paper, we present the design, implementation, and evaluation of DEEPSEC, a uniform platform that aims to bridge this gap. In its current implementation, DEEPSEC incorporates 16 state-of-the-art attacks with 10 attack utility metrics, and 13 state-of-the-art defenses with 5 defensive utility metrics. To our best knowledge, DEEPSEC is the first platform that enables researchers and practitioners to (i) measure the vulnerability of DL models, (ii) evaluate the effectiveness of various attacks/defenses, and (iii) conduct comparative studies on attacks/defenses in a comprehensive and informative manner. Leveraging DEEPSEC, we systematically evaluate the existing adversarial attack and defense methods, and draw a set of key findings, which demonstrate DEEPSEC's rich functionality, such as (1) the trade-off between misclassification and imperceptibility is empirically confirmed; (2) most defenses that claim to be universally applicable can only defend against limited types of attacks under restricted settings; (3) it is not necessary that adversarial examples with higher perturbation magnitude are easier to be detected; (4) the ensemble of multiple defenses cannot improve the overall defense capability, but can improve the lower bound of the defense effectiveness of individuals. Extensive analysis on DEEPSEC demonstrates its capabilities and advantages as a benchmark platform which can benefit future adversarial learning research.

A collaborative deep learning microservice for backdoor defenses in Industrial IoT networks

Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks

Precise Defense Approach Against Small-Scale Backdoor Attacks in Industrial Internet of Things

Edge-Cloud Collaborative Defense against Backdoor Attacks in Federated Learning

A Stability-Enhanced Dynamic Backdoor Defense in Federated Learning for IIoT

Deep Learning for Securing Software-Defined Industrial Internet of Things: Attacks and Countermeasures

A multi-point collaborative DDoS defense mechanism for IIoT environment

DeepDefense: A Steganalysis-Based Backdoor Detecting and Mitigating Protocol in Deep Neural Networks for AI Security

Secure Collaborative Deep Learning Against GAN Attacks in the Internet of Things

Stability-Based Analysis and Defense Against Backdoor Attacks on Edge Computing Services

DEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model.

DeepSweep: an Evaluation Framework for Mitigating DNN Backdoor Attacks Using Data Augmentation.

DeepGuard: Backdoor Attack Detection and Identification Schemes in Privacy-Preserving Deep Neural Networks

MT-MTD: Muti-Training Based Moving Target Defense Trojaning Attack in Edged-AI Network

Secure Deep Learning in Defense in Deep-Learning-as-a-Service Computing Systems in Digital Twins

SECaaS-based Partially Observable Defense Model for IIoT Against Advanced Persistent Threats

Backdoor Attacks and Defenses for Deep Neural Networks in Outsourced Cloud Environments

DeepDefense: Training Deep Neural Networks with Improved Robustness.

DLP: towards active defense against backdoor attacks with decoupled learning process

Stealthy and Flexible Trojan in Deep Learning Framework

Security in defect detection: A new one-pixel attack for fooling DNNs