Abstract:Transfer learning has made significant advancements, however, the issue of overfitting continues to pose a major challenge. Data augmentation has emerged as a highly promising technique to counteract this challenge. Current data augmentation methods are fixed in nature, requiring manual determination of the appropriate intensity prior to the training process. However, this entails substantial computational costs. Additionally, as the model approaches convergence, static data augmentation strategies can become suboptimal. In this paper, we introduce the concept of Dynamic Data Augmentation (DynamicAug), a method that autonomously adjusts the intensity of data augmentation, taking into account the convergence state of the model. During each iteration of the model's forward pass, we utilize a Gaussian distribution based sampler to stochastically sample the current intensity of data augmentation. To ensure that the sampled intensity is aligned with the convergence state of the model, we introduce a learnable expectation to the sampler and update the expectation iteratively. In order to assess the convergence status of the model, we introduce a novel loss function called the convergence loss. Through extensive experiments conducted over 27 vision datasets, we have demonstrated that DynamicAug can significantly enhance the performance of existing transfer learning methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to alleviate the over - fitting problem through dynamic data augmentation strategies and improve the performance of the model on specific tasks in transfer learning. Traditional data augmentation methods usually require manually determining the augmentation intensity before training, which not only increases the computational cost, but also the static data augmentation strategy may become sub - optimal when the model is close to convergence. Therefore, the author proposes a method named DynamicAug, which can automatically adjust the intensity of data augmentation according to the convergence state of the model. Specifically, DynamicAug randomly samples the current data augmentation intensity by introducing a sampler based on Gaussian distribution, and ensures that the sampling intensity is aligned with the convergence state of the model by iteratively updating the expected value of the sampler. In addition, in order to evaluate the convergence state of the model, the author introduces a new loss function - convergence loss. Through extensive experiments on 27 visual datasets, it is proved that DynamicAug can significantly improve the performance of existing transfer learning methods. ### Main Contributions 1. **Propose a dynamic model - aware augmentation method**: This method can dynamically and adaptively adjust the intensity of data augmentation according to the convergence state of the model. The performance of the model obtained by fine - tuning with dynamic data augmentation is better than that obtained by fine - tuning with static data augmentation. 2. **Wide applicability**: The DynamicAug method is not limited to the modified LoRA, Adapter and VPT methods, and can also be seamlessly combined with mainstream fine - tuning methods. 3. **Experimental verification**: The experimental results show that DynamicAug is a beneficial supplement to the current fine - tuning strategy and significantly improves the model performance. For example, on the VTAB - 1k dataset, DynamicAug improves the average accuracy of the LoRA fine - tuning method by 1.7%, and even exceeds the best fine - tuning architecture search results in NOAH and SPT. ### Key Technologies - **Dynamic data augmentation**: Dynamically adjust the data augmentation strategy by introducing a data augmentation sampler (DAS) based on Gaussian distribution. - **Convergence loss**: Introduce a new loss function to evaluate the convergence state of the model and update the intensity of data augmentation accordingly. - **Adaptive parameter update**: Embed the learnable hyper - parameter α in the forward process through the straight - through estimator (STE) and maintain its gradient in the backward propagation process. ### Experimental Setup - **Datasets**: - VTAB - 1k: It contains 19 classification tasks, which are divided into three fields: natural images, professional images and structured images. - FGVC: Fine - grained visual classification tasks, including datasets such as Stanford Dogs, Oxford Flowers, NABirds, CUB - 200 - 2011 and Stanford Cars. - Few - shot tasks: Select five fine - grained visual recognition datasets, including Food101, OxfordFlowers102, StanfordCars, OxfordPets and FGVCAircraft. - **Baseline methods**: - Use the pre - trained Vision Transformer backbone network ViT - B/16 and apply it to methods such as LoRA, Adapter and Prompt - deep to further improve performance. Through these innovations and technologies, DynamicAug shows significant performance improvement in various transfer learning tasks, especially when dealing with small - sample and complex tasks.

DynamicAug: Enhancing Transfer Learning Through Dynamic Data Augmentation Strategies Based on Model State

Boosting Unsupervised Contrastive Learning Using Diffusion-Based Data Augmentation from Scratch

Enhanced Transfer Learning with Data Augmentation

Automatic Data Augmentation by Learning the Deterministic Policy

AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation

Data-Efficient Augmentation for Training Neural Networks

Boosting Model Resilience via Implicit Adversarial Data Augmentation

DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejection

Deep AutoAugment

Accelerating Dataset Distillation Via Model Augmentation

EventAugment: Learning Augmentation Policies from Asynchronous Event-based Data

Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data

KeepAugment: A Simple Information-Preserving Data Augmentation Approach

Scale-aware Automatic Augmentations for Object Detection with Dynamic Training

LatentAugment: Dynamically Optimized Latent Probabilities of Data Augmentation

DNA: Dynamic Network Augmentation

Effective Data Augmentation With Diffusion Models

A Deep Data Augmentation Framework Based on Generative Adversarial Networks

Adaptive Data Augmentation for Contrastive Learning

A Good Data Augmentation Policy Is Not All You Need: A Multi-Task Learning Perspective