Abstract:A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the trained model tends to predict the majority of classes rather than the minority ones. That is, underfitting for minority classes seems to be one of the challenges of generalization. For a good generalization of the minority classes, we design a new Maximum Margin (MM) loss function, motivated by minimizing a margin-based generalization bound through the shifting decision bound. The theoretically-principled label-distribution-aware margin (LDAM) loss was successfully applied with prior strategies such as re-weighting or re-sampling along with the effective training schedule. However, they did not investigate the maximum margin loss function yet. In this study, we investigate the performances of two types of hard maximum margin-based decision boundary shift with LDAM's training schedule on artificially imbalanced CIFAR-10/100 for fair comparisons and effectiveness.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **the learning problem in class - imbalanced datasets**. Specifically, the trained models tend to predict the majority class rather than the minority class, leading to underfitting of the minority class, thus affecting the overall generalization ability of the model. ### Problem Background In large - scale datasets in the real world, the class distribution is usually a long - tailed distribution, that is, the number of samples in some classes is much larger than that in other classes. This imbalanced data distribution will lead to poor performance of the model on the minority class because the model is prone to over - fitting on the majority class and has insufficient learning on the minority class. ### Main Contributions of the Paper To solve this problem, the author proposes **the Maximum Margin (MM) loss function**, aiming to improve the generalization ability of the minority class by maximizing the margin of each class. Specifically: 1. **A new maximum margin loss function is designed**: This loss function encourages the model to have a larger margin on the minority class, thereby improving the classification accuracy of the minority class. 2. **The Deferred Re - balancing Optimization Schedule is applied**: Combined with hard positive/negative margins to achieve better generalization effects. 3. **Experimental verification**: Experiments were carried out on artificially generated imbalanced CIFAR - 10 and CIFAR - 100 datasets and compared with existing methods, demonstrating its effectiveness. ### Key Formulas To better understand the design of the maximum margin loss function, here are some key formulas mentioned in the paper: - Hard Positive Margin: \[ \Delta^+_y=\exp\left(-\max(z_y-\max_{j\neq y}z_j, 0)-\delta^+\right) \] - Hard Negative Margin: \[ \Delta^-_y=\exp\left(-\max(\max_{j\neq y}z_j - z_y, 0)-\delta^-\right) \] - Maximum Margin Loss: \[ L_{MM}((x, y); f)=-\log\frac{e^{z_y-\Delta^{MM}_y}}{e^{z_y-\Delta^{MM}_y}+\sum_{j\neq y}e^{z_j}} \] where \(\Delta^{MM}_y\) is defined according to formula (1). Through these formulas, the model can dynamically adjust the decision boundary during the training process, so that the samples of the minority class receive more attention, thereby improving the overall generalization ability.

Learning Imbalanced Datasets with Maximum Margin Loss

Towards Desirable Decision Boundary by Moderate-Margin Adversarial Training

Margin-Aware Adaptive-Weighted-Loss for deep learning based imbalanced data classification

Learning Towards The Largest Margins.

Enlarged Large Margin Loss for Imbalanced Classification

MMD-encouraging convolutional autoencoder: a novel classification algorithm for imbalanced data

Borderline-margin loss based deep metric learning framework for imbalanced data

Value-Aware Resampling and Loss for Imbalanced Classification

The Ensemble of Density-Sensitive SVDD Classifier Based on Maximum Soft Margin for Imbalanced Datasets.

Margin Distribution Analysis

MMA Training: Direct Input Space Margin Maximization through Adversarial Training

Large Margin Discriminative Loss for Classification

Adaptive maximum margin criterion for image classification

Maximum Margin Multi-Dimensional Classification

IMMC: Incremental Maximum Margin Criterion.

Multiset Feature Learning for Highly Imbalanced Data Classification

Adversarial Margin Maximization Networks

A Family of Maximum Margin Criterion for Adaptive Learning

LAM3L: Locally Adaptive Maximum Margin Metric Learning for Visual Data Classification

Identifying Mislabeled Data using the Area Under the Margin Ranking

Margin Calibration for Long-Tailed Visual Recognition