Abstract:Adversarial examples pose a significant challenge to the security of deep neural networks (DNNs). In order to defend against malicious attacks, adversarial training forces DNNs to learn more robust features by suppressing generalizable but non-robust features, which boosts the robustness while suffering from significant accuracy drops on clean images. Ensemble training, on the other hand, trains multiple sub-models to predict data for improved robustness and still achieves desirable accuracy on clean data. Despite these efforts, previous ensemble methods are still susceptible to attacks and fail to increase model diversity as the size of the ensemble group increases. In this work, we revisit the model diversity from the perspective of data and discover that high similarity between training batches decreases feature diversity and weakens ensemble robustness. To this end, we propose La tent Fe ature D iversification (LAFED) , which reconstructs training sets with diverse features during the optimization, enhancing the overall robustness of an ensemble. For each sub-model, LAFED treats the vulnerability extracted from other sub-models as raw data, which is then combined with round-changed weights with a stochastic manner in the latent space. This results in the formation of new features, remarkably reducing the similarity of learned representations between the sub-models. Furthermore, LAFED enhances feature diversity within the ensemble model by utilizing hierarchical smoothed labels. Extensive experiments illustrate that LAFED significantly improves diversity among sub-models and enhances robustness against adversarial attacks compared to current methods. The code is publicly available at https://github.com/zhuangwz/LAFED .

LAFED: Towards robust ensemble models via latent feature diversification

Diversity supporting robustness: Enhancing adversarial robustness via differentiated ensemble predictions

DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles

Improving Adversarial Robustness via Promoting Ensemble Diversity.

Improving Adversarial Robustness Via Promoting Ensemble Diversity

Ensemble Adversarial Defense via Integration of Multiple Dispersed Low Curvature Models

Ensemble Federated Adversarial Training with Non-IID data

Exploring Model Learning Heterogeneity for Boosting Ensemble Robustness

Feature Augmentation for Adversarial Robustness

Learning Diverse Models for End-to-End Ensemble Tracking.

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

Combating Exacerbated Heterogeneity for Robust Models in Federated Learning

On the Certified Robustness for Ensemble Models and Beyond

Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustness

Combating Exacerbated Heterogeneity for Robust Decentralized Models

Online Knowledge Distillation via Multi-branch Diversity Enhancement

AFD: Mitigating Feature Gap for Adversarial Robustness by Feature Disentanglement

Efficient Diversity-Driven Ensemble for Deep Neural Networks

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning