Abstract:Deep learning has revolutionized the performance of classification, but meanwhile demands sufficient labeled data for training. Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else. Such regimes arise from practical situations where not only data labeling but also data collection itself is expensive. We propose a deep adversarial data augmentation (DADA) technique to address the problem, in which we elaborately formulate data augmentation as a problem of training a class-conditional and supervised generative adversarial network (GAN). Specifically, a new discriminator loss is proposed to fit the goal of data augmentation, through which both real and augmented samples are enforced to contribute to and be consistent in finding the decision boundaries. Tailored training techniques are developed accordingly. To quantitatively validate its effectiveness, we first perform extensive simulations to show that DADA substantially outperforms both traditional data augmentation and a few GAN-based options. We then extend experiments to three real-world small labeled datasets where existing data augmentation and/or transfer learning strategies are either less effective or infeasible. All results endorse the superior capability of DADA in enhancing the generalization ability of deep networks trained in practical extremely low data regimes. Source code is available at <a class="link-external link-https" href="https://github.com/SchafferZhang/DADA" rel="external noopener nofollow">this https URL</a>.

Rethinking Cost-sensitive Classification in Deep Learning via Adversarial Data Augmentation

A Cost-Sensitive Deep Learning-Based Approach for Network Traffic Classification

Boosting Adversarial Training in Safety-Critical Systems Through Boundary Data Selection

Improving Adversarial Robustness of 3D Point Cloud Classification Models

Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation

Adaptive Cost-Sensitive Learning in Neural Networks for Misclassification Cost Problems

An adaptive cost-sensitive learning approach in neural networks to minimize local training–test class distributions mismatch

Cost-Sensitive Robustness against Adversarial Examples

Adaptive Cost-Sensitive Online Classification

Decision Boundary-Aware Data Augmentation for Adversarial Training

DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification

Boosting Model Resilience via Implicit Adversarial Data Augmentation

Understanding the Detrimental Class-level Effects of Data Augmentation

Cost-Sensitive Alternating Direction Method Of Multipliers For Large-Scale Classification

Learning Only When It Matters: Cost-Aware Long-Tailed Classification

Improving Adversarial Robustness Requires Revisiting Misclassified Examples.

Reducing Adversarial Training Cost with Gradient Approximation

Mitigating Adversarial Attacks for Deep Neural Networks by Input Deformation and Augmentation

Online classifier adaptation for cost-sensitive learning

Class-aware domain adaptation for improving adversarial robustness