Abstract:Improving the classification of multi-class imbalanced data is more difficult than its two-class counterpart. In this paper, we use deep neural networks to train new representations of tabular multi-class data. Unlike the typically developed re-sampling pre-processing methods, our proposal modifies the distribution of features, i.e. the positions of examples in the learned embedded representation, and it does not modify the class sizes. To learn such embedded representations we introduced various definitions of triplet loss functions: the simplest one uses weights related to the degree of class imbalance, while the next proposals are intended for more complex distributions of examples and aim to generate a safe neighborhood of minority examples. Similarly to the resampling approaches, after applying such preprocessing, different classifiers can be trained on new representations. Experiments with popular multi-class imbalanced benchmark data sets and three classifiers showed the advantage of the proposed approach over popular pre-processing methods as well as basic versions of neural networks with classical loss function formulations.

What problem does this paper attempt to address?

This paper attempts to solve the classification problem of multi - class imbalanced data, especially for tabular data. Compared with the binary - class imbalance problem, the multi - class imbalance problem is more complex and difficult because the relationships between different classes are more complicated and the learning difficulty of a single sample is also greater. To solve these problems, the author proposes a new method based on deep neural networks, which learns new feature representations by modifying the triplet loss function instead of resampling the data as in traditional methods. Specifically, the main contributions of the paper include: 1. **Introduction and improvement of the triplet loss function**: To deal with the class - imbalance problem, the author proposes several new variants of the triplet loss function to better handle minority - class samples and consider local data - difficulty factors. 2. **Learning of deep neural networks**: Use the proposed triplet loss function to train deep neural networks to generate new embedding representations, thereby improving the classification performance of multi - class imbalanced data. 3. **Experimental verification**: Through experiments on several commonly - used multi - class imbalanced datasets, the performance of the new method is compared with that of existing pre - processing methods and classical neural networks, demonstrating the advantages of the new method. ### Core problems of the paper - **Difficulty in classifying multi - class imbalanced data**: The classification of multi - class imbalanced data is more difficult than the binary - class problem because the relationships between classes are more complex and the local data - difficulty is greater. - **Limitations of existing methods**: Traditional resampling methods or simple loss - function adjustments have limited effectiveness in dealing with multi - class imbalance problems. ### Solutions - **New triplet loss function**: By introducing a weighted triplet loss function, minority - class samples receive more attention during the learning process, thereby improving the classifier's ability to recognize minority classes. - **Local neighborhood security**: Consider the sample distribution within the local neighborhood to ensure that minority - class samples have a sufficient safety distance around them and avoid being overwhelmed by majority - class samples. ### Experimental results The paper verifies the effectiveness of the new method through experiments, especially outperforming existing pre - processing methods and classical neural networks on multiple benchmark datasets. In addition, visual analysis also shows that the new embedding representation can better separate samples of different classes, especially on some complex multi - class imbalanced datasets. In conclusion, this paper aims to effectively solve the classification problem of multi - class imbalanced data by improving the triplet loss function and using deep neural networks to learn new feature representations.

Deep Similarity Learning Loss Functions in Data Transformation for Class Imbalance

Towards Deeper Insights into Deep Learning from Imbalanced Data.

An Adaptive Deep Metric Learning Loss Function for Class-Imbalance Learning via Intraclass Diversity and Interclass Distillation

Leveraging Class Similarity to Improve Deep Neural Network Robustness

Ranked List Loss for Deep Metric Learning

Value-Aware Resampling and Loss for Imbalanced Classification

Learning Deep Landmarks for Imbalanced Classification

Margin-Aware Adaptive-Weighted-Loss for deep learning based imbalanced data classification

Borderline-margin loss based deep metric learning framework for imbalanced data

Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning

Dual Compensation Residual Networks for Class Imbalanced Learning

Improving the Accuracy of Learning Example Weights for Imbalance Classification

Deep Metric Learning with Tuplet Margin Loss.

Procrustean Training for Imbalanced Deep Learning

OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning

Phased progressive learning with coupling-regulation-imbalance loss for imbalanced data classification

Survey on deep learning with class imbalance

Towards Effective Collaborative Learning in Long-Tailed Recognition

Introducing DeepBalance: Random Deep Belief Network Ensembles to Address Class Imbalance

Similarity-Dissimilarity Loss with Supervised Contrastive Learning for Multi-label Classification

Self-paced Ensemble for Highly Imbalanced Massive Data Classification