Abstract:Improving the generalization of deep networks is an important open challenge, particularly in domains without plentiful data. The mixup algorithm improves generalization by linearly interpolating a pair of examples and their corresponding labels. These interpolated examples augment the original training set. Mixup has shown promising results in various classification tasks, but systematic analysis of mixup in regression remains underexplored. Using mixup directly on regression labels can result in arbitrarily incorrect labels. In this paper, we propose a simple yet powerful algorithm, C-Mixup, to improve generalization on regression tasks. In contrast with vanilla mixup, which picks training examples for mixing with uniform probability, C-Mixup adjusts the sampling probability based on the similarity of the labels. Our theoretical analysis confirms that C-Mixup with label similarity obtains a smaller mean square error in supervised regression and meta-regression than vanilla mixup and using feature similarity. Another benefit of C-Mixup is that it can improve out-of-distribution robustness, where the test distribution is different from the training distribution. By selectively interpolating examples with similar labels, it mitigates the effects of domain-associated information and yields domain-invariant representations. We evaluate C-Mixup on eleven datasets, ranging from tabular to video data. Compared to the best prior approach, C-Mixup achieves 6.56%, 4.76%, 5.82% improvements in in-distribution generalization, task generalization, and out-of-distribution robustness, respectively. Code is released at <a class="link-external link-https" href="https://github.com/huaxiuyao/C-Mixup" rel="external noopener nofollow">this https URL</a>.

Mixup in Latent Geometry for Graph Classification

GeoMix: Towards Geometry-Aware Data Augmentation

AGMixup: Adaptive Graph Mixup for Semi-supervised Node Classification

GraphMAD: Graph Mixup for Data Augmentation using Data-Driven Convex Clustering

ifMixup: Interpolating Graph Pair to Regularize Graph Classification

Efficient and Effective Augmentation Framework with Latent Mixup and Label-Guided Contrastive Learning for Graph Classification

Decoupled Mixup for Data-efficient Learning

Understanding Mixup Training Methods

Global Mixup: Eliminating Ambiguity with Clustering.

MixupE: Understanding and Improving Mixup from Directional Derivative Perspective

Fused Gromov-Wasserstein Graph Mixup for Graph-level Classifications

Mixup Augmentation with Multiple Interpolations

GraphMixup: Improving Class-Imbalanced Node Classification on Graphs by Self-supervised Context Prediction

C-Mixup: Improving Generalization in Regression

IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors

Unveiling the Power of Mixup for Stronger Classiﬁers

Multi-scale Self-Attention Mixup for Graph Classification

On the Equivalence of Graph Convolution and Mixup

MixUp As Locally Linear Out-of-manifold Regularization

R-Mixup: Riemannian Mixup for Biological Networks

Mixup Without Hesitation