Abstract:Recent research on robustness has revealed significant performance gaps between neural image classifiers trained on datasets that are similar to the test set, and those that are from a naturally shifted distribution, such as sketches, paintings, and animations of the object categories observed during training. Prior work focuses on reducing this gap by designing engineered augmentations of training data or through unsupervised pretraining of a single large model on massive in-the-wild training datasets scraped from the Internet. However, the notion of a dataset is also undergoing a paradigm shift in recent years. With drastic improvements in the quality, ease-of-use, and access to modern generative models, generated data is pervading the web. In this light, we study the question: How do these generated datasets influence the natural robustness of image classifiers? We find that Imagenet classifiers trained on real data augmented with generated data achieve higher accuracy and effective robustness than standard training and popular augmentation strategies in the presence of natural distribution shifts. We analyze various factors influencing these results, including the choice of conditioning strategies and the amount of generated data. Additionally, we find that the standard ImageNet classifiers suffer a performance degradation of upto 20\% on the generated data, indicating their fragility at accurately classifying the objects under novel variations. Lastly, we demonstrate that the image classifiers, which have been trained on real data augmented with generated data from the base generative model, exhibit greater resilience to natural distribution shifts compared to the classifiers trained on real data augmented with generated data from the finetuned generative model on the real data. The code, models, and datasets are available at <a class="link-external link-https" href="https://github.com/Hritikbansal/generative-robustness" rel="external noopener nofollow">this https URL</a>.

Towards Understanding the Generative Capability of Adversarially Robust Classifiers

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training

Exploring the Connection between Robust and Generative Models

Shedding More Light on Robust Classifiers under the lens of Energy-based Models

Adversarial Training Improves Joint Energy-Based Generative Modelling

Towards Bridging the Performance Gaps of Joint Energy-based Models

Adversarial Code Learning for Image Generation

Counterfactual Image Generation for adversarially robust and interpretable Classifiers

The Adversarial AI-Art: Understanding, Generation, Detection, and Benchmarking

EGC: Image Generation and Classification via a Diffusion Energy-Based Model

EGM: an Efficient Generative Model for Unrestricted Adversarial Examples

Energy-based Generative Adversarial Network

GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models

Leaving Reality to Imagination: Robust Classification via Generated Datasets

Adversarial Attack Against Images Classification based on Generative Adversarial Networks

Leveraging generative models to characterize the failure conditions of image classifiers

Adversarially Robust Classification by Conditional Generative Model Inversion

A Generative Adversarial Network Based on Energy Function

Adversarial Image Generation and Training for Deep Convolutional Neural Networks.