Abstract:Computer vision models normally witness degraded performance when deployed in real-world scenarios, due to unexpected changes in inputs that were not accounted for during training. Data augmentation is commonly used to address this issue, as it aims to increase data variety and reduce the distribution gap between training and test data. However, common visual augmentations might not guarantee extensive robustness of computer vision models. In this paper, we propose Auxiliary Fourier-basis Augmentation (AFA), a complementary technique targeting augmentation in the frequency domain and filling the augmentation gap left by visual augmentations. We demonstrate the utility of augmentation via Fourier-basis additive noise in a straightforward and efficient adversarial setting. Our results show that AFA benefits the robustness of models against common corruptions, OOD generalization, and consistency of performance of models against increasing perturbations, with negligible deficit to the standard performance of models. It can be seamlessly integrated with other augmentation techniques to further boost performance. Code and models can be found at:

What problem does this paper attempt to address?

### The Problem Addressed by the Paper This paper aims to address the issue of performance degradation encountered by computer vision models when deployed in real-world scenarios. Specifically, the performance of the model is affected when the input data exhibits variations that were not considered during training. Although data augmentation techniques are widely used to increase data diversity and reduce the distribution gap between training and testing data, common visual augmentation techniques may not ensure broad robustness of the model. To this end, the authors propose a new augmentation technique—Auxiliary Fourier-basis Augmentation (AFA). This technique supplements existing visual augmentation methods by adding Fourier basis function noise in the frequency domain, thereby filling the robustness gap left by visual augmentations. AFA not only improves the model's robustness to common image corruptions but also enhances the model's OOD generalization ability and consistency against adversarial perturbations, with minimal impact on standard performance. Additionally, AFA can be seamlessly integrated with other augmentation techniques to further enhance model performance. ### Main Contributions 1. **Proposed a simple and computationally efficient augmentation technique**: AFA enhances model robustness by adding Fourier basis function noise in the frequency domain, improving the model's resistance to common image corruptions, OOD generalization ability, and consistency against adversarial perturbations. 2. **Expanded the augmentation space**: By utilizing frequency noise with adjustable amplitude and phase, and employing these noises in adversarial settings, AFA supplements the shortcomings of existing visual augmentation techniques, reducing the gap left by visual augmentations. 3. **High computational efficiency**: AFA requires only one additional step during training, without the need for multiple preprocessing steps and expensive computations like other methods. This allows AFA to be trained on larger models and datasets, such as ImageNet. ### Experimental Results 1. **Comparison with other augmentation techniques**: AFA outperforms or at least matches the performance of existing augmentation techniques on multiple benchmark datasets, with particularly outstanding performance on ImageNet. 2. **Robustness to severe corruptions**: AFA shows consistent improvements under different severity levels of corruption, with more significant improvements under higher severity levels. 3. **Spectral robustness evaluation**: Through Fourier heatmaps, the authors demonstrate AFA's robustness to perturbations at specific frequencies, validating the effectiveness of AFA. In summary, this paper effectively addresses the robustness issue of computer vision models in real-world deployments by introducing the AFA technique, providing new insights for improving model reliability and generalization ability.

Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification

Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration

A Fourier Perspective On Model Robustness In Computer Vision

DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learning

Feature Augmentation for Adversarial Robustness

HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness

Fourier-based augmentation with applications to domain generalization

Frequency Domain Model Augmentation for Adversarial Attack

Data Augmentation Can Improve Robustness

Adversarial Feature Augmentation and Normalization for Visual Recognition

FrAug: Frequency Domain Augmentation for Time Series Forecasting

Disentangling the Effects of Data Augmentation and Format Transform in Self-Supervised Learning of Image Representations

Improving Model Generalization by On-manifold Adversarial Augmentation in the Frequency Domain

CamoFA: A Learnable Fourier-based Augmentation for Camouflage Segmentation

Multi-Frequency Augmentation framework via information active capture for machinery intelligent fault diagnosis

Boosting Model Resilience via Implicit Adversarial Data Augmentation

Soft Augmentation for Image Classification

DiffAug: A Diffuse-and-Denoise Augmentation for Training Robust Classifiers

Data-Efficient Augmentation for Training Neural Networks

Augmentation Inside the Network

DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejection