Abstract:Adversarial examples pose a significant challenge to the security of deep neural networks (DNNs). In order to defend against malicious attacks, adversarial training forces DNNs to learn more robust features by suppressing generalizable but non-robust features, which boosts the robustness while suffering from significant accuracy drops on clean images. Ensemble training, on the other hand, trains multiple sub-models to predict data for improved robustness and still achieves desirable accuracy on clean data. Despite these efforts, previous ensemble methods are still susceptible to attacks and fail to increase model diversity as the size of the ensemble group increases. In this work, we revisit the model diversity from the perspective of data and discover that high similarity between training batches decreases feature diversity and weakens ensemble robustness. To this end, we propose La tent Fe ature D iversification (LAFED) , which reconstructs training sets with diverse features during the optimization, enhancing the overall robustness of an ensemble. For each sub-model, LAFED treats the vulnerability extracted from other sub-models as raw data, which is then combined with round-changed weights with a stochastic manner in the latent space. This results in the formation of new features, remarkably reducing the similarity of learned representations between the sub-models. Furthermore, LAFED enhances feature diversity within the ensemble model by utilizing hierarchical smoothed labels. Extensive experiments illustrate that LAFED significantly improves diversity among sub-models and enhances robustness against adversarial attacks compared to current methods. The code is publicly available at https://github.com/zhuangwz/LAFED .

Exploiting Unlabeled Data to Enhance Ensemble Diversity

Clustering Ensemble with High Diversity Based on Adding Artificial Data

Efficient Diversity-Driven Ensemble for Deep Neural Networks

Something for (almost) nothing: Improving deep ensemble calibration using unlabeled data

Scalable Ensemble Diversification for OOD Generalization and Detection

When Semi-supervised Learning Meets Ensemble Learning

A hybrid data-level ensemble to enable learning from highly imbalanced dataset

Promoting High Diversity Ensemble Learning with EnsembleBench

LAFED: Towards robust ensemble models via latent feature diversification

A Unified Theory of Diversity in Ensemble Learning

Developing parsimonious ensembles using ensemble diversity within a reinforcement learning framework

Learning to Diversify via Weighted Kernels for Classifier Ensemble

Unsupervised Ensemble Learning Via Network Generation.

Data Augmentation For Label Enhancement

Dynamic Ensemble Learning for Multi-label Classification

Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination

Ensemble Learning through Diversity Management: Theory, Algorithms, and Applications

Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

Diversity Helps: Unsupervised Few-shot Learning Via Distribution Shift-based Data Augmentation

Diversity Induced Environment Design via Self-Play

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity