Abstract:Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years. However, the progress of building more robust models is usually hampered by the incomplete or incorrect robustness evaluation. To accelerate the research on reliable evaluation of adversarial robustness of the current defense models in image classification, the TSAIL group at Tsinghua University and the Alibaba Security group organized this competition along with a CVPR 2021 workshop on adversarial machine learning (<a class="link-external link-https" href="https://aisecure-workshop.github.io/amlcvpr2021/" rel="external noopener nofollow">this https URL</a>). The purpose of this competition is to motivate novel attack algorithms to evaluate adversarial robustness more effectively and reliably. The participants were encouraged to develop stronger white-box attack algorithms to find the worst-case robustness of different defenses. This competition was conducted on an adversarial robustness evaluation platform -- ARES (<a class="link-external link-https" href="https://github.com/thu-ml/ares" rel="external noopener nofollow">this https URL</a>), and is held on the TianChi platform (<a class="link-external link-https" href="https://tianchi.aliyun.com/competition/entrance/531847/introduction" rel="external noopener nofollow">this https URL</a>) as one of the series of AI Security Challengers Program. After the competition, we summarized the results and established a new adversarial robustness benchmark at <a class="link-external link-https" href="https://ml.cs.tsinghua.edu.cn/ares-bench/" rel="external noopener nofollow">this https URL</a>, which allows users to upload adversarial attack algorithms and defense models for evaluation.

CMMR: A Composite Multidimensional Models Robustness Evaluation Framework for Deep Learning.

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Improving Adversarial Robustness of 3D Point Cloud Classification Models

ROBY: Evaluating the adversarial robustness of a deep model by its decision boundaries

MultiRobustBench: Benchmarking Robustness Against Multiple Attacks

Mutual Learning-Based Framework for Enhancing Robustness of Code Models Via Adversarial Training

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework

MMAD-Purify: A Precision-Optimized Framework for Efficient and Scalable Multi-Modal Attacks

MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks

Understanding and Measuring Robustness of Multimodal Learning

Impact of Architectural Modifications on Deep Learning Adversarial Robustness

Robust Mode Connectivity-Oriented Adversarial Defense: Enhancing Neural Network Robustness Against Diversified $\ell_p$ Attacks

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

Robustness-Eva-MRC: Assessing and Analyzing the Robustness of Neural Models in Extractive Machine Reading Comprehension

Composite Adversarial Attacks

Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume

ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries

Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Models

Adversarial Attacks on ML Defense Models Competition

MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models