Abstract:Deep learning has revolutionized the performance of classification, but meanwhile demands sufficient labeled data for training. Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else. Such regimes arise from practical situations where not only data labeling but also data collection itself is expensive. We propose a deep adversarial data augmentation (DADA) technique to address the problem, in which we elaborately formulate data augmentation as a problem of training a class-conditional and supervised generative adversarial network (GAN). Specifically, a new discriminator loss is proposed to fit the goal of data augmentation, through which both real and augmented samples are enforced to contribute to and be consistent in finding the decision boundaries. Tailored training techniques are developed accordingly. To quantitatively validate its effectiveness, we first perform extensive simulations to show that DADA substantially outperforms both traditional data augmentation and a few GAN-based options. We then extend experiments to three real-world small labeled datasets where existing data augmentation and/or transfer learning strategies are either less effective or infeasible. All results endorse the superior capability of DADA in enhancing the generalization ability of deep networks trained in practical extremely low data regimes. Source code is available at <a class="link-external link-https" href="https://github.com/SchafferZhang/DADA" rel="external noopener nofollow">this https URL</a>.

Adaptive Noisy Data Augmentation for Regularized Estimation and Inference of Generalized Linear Models

Adaptive Noisy Data Augmentation for Regularized Estimation and Inference in Generalized Linear Models

PANDA: AdaPtive Noisy Data Augmentation for Regularization of Undirected Graphical Models

AdaPtive Noisy Data Augmentation (PANDA) for Simultaneous Construction of Multiple Graph Models

Improving Data Analytics with Fast and Adaptive Regularization

Adaptive Lightweight Regularization Tool for Complex Analytics

PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs

Flexible, non-parametric modeling using regularized neural networks

A Regularization-Based Adaptive Test for High-Dimensional Generalized Linear Models

Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural Networks

A Perturbation Method for Inference on Regularized Regression Estimates

Adaptive Data Augmentation for Supervised Learning over Missing Data

A Note on Adaptive Lp Regularization

GSDAR: a Fast Newton Algorithm for ℓ _0 Regularized Generalized Linear Models with Statistical Guarantee

A generalization of regularized dual averaging and its dynamics

Versatile Descent Algorithms for Group Regularization and Variable Selection in Generalized Linear Models

Adaptive debiased SGD in high-dimensional GLMs with streaming data

PDA: Progressive Data Augmentation for General Robustness of Deep Neural Networks

DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification

AdaL: Adaptive Gradient Transformation Contributes to Convergences and Generalizations

Generalized Partial Linear Models with Nonignorable Dropouts