Abstract:Data augmentation is widely known as a simple yet surprisingly effective technique for regularizing deep networks. Conventional data augmentation schemes, e.g., flipping, translation or rotation, are low-level, data-independent and class-agnostic operations, leading to limited diversity for augmented samples. To this end, we propose a novel semantic data augmentation algorithm to complement traditional approaches. The proposed method is inspired by the intriguing property that deep networks are effective in learning linearized features, i.e., certain directions in the deep feature space correspond to meaningful semantic transformations, e.g., changing the background or view angle of an object. Based on this observation, translating training samples along many such directions in the feature space can effectively augment the dataset for more diversity. To implement this idea, we first introduce a sampling based method to obtain semantically meaningful directions efficiently. Then, an upper bound of the expected cross-entropy (CE) loss on the augmented training set is derived by assuming the number of augmented samples goes to infinity, yielding a highly efficient algorithm. In fact, we show that the proposed implicit semantic data augmentation (ISDA) algorithm amounts to minimizing a novel robust CE loss, which adds minimal extra computational cost to a normal training procedure. In addition to supervised learning, ISDA can be applied to semi-supervised learning tasks under the consistency regularization framework, where ISDA amounts to minimizing the upper bound of the expected KL-divergence between the augmented features and the original features. Although being simple, ISDA consistently improves the generalization performance of popular deep models (e.g., ResNets and DenseNets) on a variety of datasets, i.e., CIFAR-10, CIFAR-100, SVHN, ImageNet, and Cityscapes.

Semantic Augmentation in Images using Language

Boosting Unsupervised Contrastive Learning Using Diffusion-Based Data Augmentation from Scratch

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Semantic-Guided Generative Image Augmentation Method with Diffusion Models for Image Classification

Effective Data Augmentation With Diffusion Models

Semantic-aware Data Augmentation for Text-to-image Synthesis

Regularizing Deep Networks with Semantic Data Augmentation

DIAGen: Diverse Image Augmentation with Generative Models

Image Data Augmentation for Deep Learning: A Survey

A survey on Image Data Augmentation for Deep Learning

Inter-Class and Inter-Domain Semantic Augmentation for Domain Generalization

Cap2Aug: Caption guided Image to Image data Augmentation

Augmentation-based Domain Generalization for Semantic Segmentation

Enhanced Generative Data Augmentation for Semantic Segmentation via Stronger Guidance

SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation

3D-VirtFusion: Synthetic 3D Data Augmentation through Generative Diffusion Models and Controllable Editing

Augmentation Policy Generation for Image Classification Using Large Language Models

Data Augmentation for Image Classification using Generative AI

Deep Visual Semantic Embedding with Text Data Augmentation and Word Embedding Initialization

Fine-Grained Recognition With Learnable Semantic Data Augmentation