Abstract:Adversarial robustness of deep learning models has gained much traction in the last few years. Various attacks and defenses are proposed to improve the adversarial robustness of modern-day deep learning architectures. While all these approaches help improve the robustness, one promising direction for improving adversarial robustness is unexplored, i.e., the complex topology of the neural network architecture. In this work, we address the following question: Can the complex topology of a neural network give adversarial robustness without any form of adversarial training?. We answer this empirically by experimenting with different hand-crafted and NAS-based architectures. Our findings show that, for small-scale attacks, NAS-based architectures are more robust for small-scale datasets and simple tasks than hand-crafted architectures. However, as the size of the dataset or the complexity of task increases, hand-crafted architectures are more robust than NAS-based architectures. Our work is the first large-scale study to understand adversarial robustness purely from an architectural perspective. Our study shows that random sampling in the search space of DARTS (a popular NAS method) with simple ensembling can improve the robustness to PGD attack by nearly~12\%. We show that NAS, which is popular for achieving SoTA accuracy, can provide adversarial accuracy as a free add-on without any form of adversarial training. Our results show that leveraging the search space of NAS methods with methods like ensembles can be an excellent way to achieve adversarial robustness without any form of adversarial training. We also introduce a metric that can be used to calculate the trade-off between clean accuracy and adversarial robustness. Code and pre-trained models will be made available at \url{<a class="link-external link-https" href="https://github.com/tdchaitanya/nas-robustness" rel="external noopener nofollow">this https URL</a>}

Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities

Discovering Robust Convolutional Architecture at Targeted Capacity: A Multi-Shot Approach

Prioritized Architecture Sampling with Monto-Carlo Tree Search

One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting

Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search

When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks

MNGNAS: Distilling Adaptive Combination of Multiple Searched Networks for One-Shot Neural Architecture Search

Single Path One-Shot Neural Architecture Search with Uniform Sampling

Balanced One-shot Neural Architecture Optimization

On Adversarial Robustness: A Neural Architecture Search perspective

BayesNAS: A Bayesian Approach for Neural Architecture Search

NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization

XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks

Efficient Novelty-Driven Neural Architecture Search

Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness

One-Shot Neural Architecture Search Through A Posteriori Distribution Guided Sampling.

GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet

AttentiveNAS: Improving Neural Architecture Search Via Attentive Sampling

DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation