Abstract:Deep learning (DL) models are inherently vulnerable to adversarial examples - maliciously crafted inputs to trigger target DL models to misbehave - which significantly hinders the application of DL in security-sensitive domains. Intensive research on adversarial learning has led to an arms race between adversaries and defenders. Such plethora of emerging attacks and defenses raise many questions: Which attacks are more evasive, preprocessing-proof, or transferable? Which defenses are more effective, utility-preserving, or general? Are ensembles of multiple defenses more robust than individuals? Yet, due to the lack of platforms for comprehensive evaluation on adversarial attacks and defenses, these critical questions remain largely unsolved. In this paper, we present the design, implementation, and evaluation of DEEPSEC, a uniform platform that aims to bridge this gap. In its current implementation, DEEPSEC incorporates 16 state-of-the-art attacks with 10 attack utility metrics, and 13 state-of-the-art defenses with 5 defensive utility metrics. To our best knowledge, DEEPSEC is the first platform that enables researchers and practitioners to (i) measure the vulnerability of DL models, (ii) evaluate the effectiveness of various attacks/defenses, and (iii) conduct comparative studies on attacks/defenses in a comprehensive and informative manner. Leveraging DEEPSEC, we systematically evaluate the existing adversarial attack and defense methods, and draw a set of key findings, which demonstrate DEEPSEC's rich functionality, such as (1) the trade-off between misclassification and imperceptibility is empirically confirmed; (2) most defenses that claim to be universally applicable can only defend against limited types of attacks under restricted settings; (3) it is not necessary that adversarial examples with higher perturbation magnitude are easier to be detected; (4) the ensemble of multiple defenses cannot improve the overall defense capability, but can improve the lower bound of the defense effectiveness of individuals. Extensive analysis on DEEPSEC demonstrates its capabilities and advantages as a benchmark platform which can benefit future adversarial learning research.

DEEPJUDGE: A Testing Framework for Copyright Protection of Deep Learning Models.

Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models

A Deep Learning Framework Supporting Model Ownership Protection and Traitor Tracing

DeepDIST: A Black-Box Anti-Collusion Framework for Secure Distribution of Deep Models

DeepTaster: Adversarial Perturbation-Based Fingerprinting to Identify Proprietary Dataset Use in Deep Neural Networks

DeepCore: Simple Fingerprint Construction for Differentiating Homologous and Piracy Models

How to Prove Your Model Belongs to You: A Blind-Watermark based Framework to Protect Intellectual Property of DNN

DE-COP: Detecting Copyrighted Content in Language Models Training Data

The Boundaries of Copyright Protection for Deep Learning Technologies of Artificial Intelligence

VeriDIP: Verifying Ownership of Deep Neural Networks through Privacy Leakage Fingerprints

Deep Intellectual Property: A Survey

Deep Model Intellectual Property Protection Via Deep Watermarking

Protecting Deep Learning Model Copyrights with Adversarial Example-Free Reuse Detection

DEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model.

RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model

CopyLens: Dynamically Flagging Copyrighted Sub-Dataset Contributions to LLM Outputs

Turn Passive to Active: A Survey on Active Intellectual Property Protection of Deep Learning Models

Copyright Protection in Generative AI: A Technical Perspective

CopyScope: Model-level Copyright Infringement Quantification in the Diffusion Workflow

Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations

Intellectual Property Protection for Deep Learning Model and Dataset Intelligence