Abstract:Owing to the superior performances, exemplar-based methods with knowledge distillation (KD) are widely applied in class incremental learning (CIL). However, it suffers from two drawbacks: 1) data imbalance between the old/learned and new classes causes the bias of the new classifier toward the head/new classes and 2) deep neural networks (DNNs) suffer from distribution drift when learning sequence tasks, which results in narrowed feature space and deficient representation of old tasks. For the first problem, we analyze the insufficiency of softmax loss when dealing with the problem of data imbalance in theory and then propose the imbalance softmax (im-softmax) loss to relieve the imbalanced data learning, where we re-scale the output logits to underfit the head/new classes. For another problem, we calibrate the feature space by incremental-adaptive angular margin (IAAM) loss. The new classes form a complete distribution in feature space yet the old are squeezed. To recover the old feature space, we first compute the included angle of normalized features and normalized anchor prototypes, and use the angle distribution to represent the class distribution, then we replenish the old distribution with the deviation from the new. Each anchor prototype is predefined as a learnable vector for a designated class. The proposed im-softmax reduces the bias in the linear classification layer. IAAM rectifies the representation learning, reduces the intra-class distance, and enlarges the inter-class margin. Finally, we seamlessly combine the im-softmax and IAAM in an end-to-end training framework, called the dual balanced class incremental learning (DBL), for further improvements. Experiments demonstrate the proposed method achieves state-of-the-art (SOTA) performances on several benchmarks, such as CIFAR10, CIFAR100, Tiny-ImageNet, and ImageNet-100.

Dual Autoencoders Features for Imbalance Classification Problem.

Learning Robust Features with Incremental Auto-Encoders

A Label Noise Robust Stacked Auto-Encoder Algorithm for Inaccurate Supervised Classification Problems

Representation Learning Via a Semi-Supervised Stacked Distance Autoencoder for Image Classification

ABAE: Auxiliary Balanced AutoEncoder for class-imbalanced semi-supervised learning

Representation Learning With Dual Autoencoder for Multi-Label Classification

Credit Card Fraud Detection Using Autoencoder Neural Network

MMD-encouraging convolutional autoencoder: a novel classification algorithm for imbalanced data

IMBALANCED DATA CLASSIFICATION BASED ON EXTREME LEARNING MACHINE AUTOENCODER

Stacked Similarity-Aware Autoencoders.

Latent Vector Expansion using Autoencoder for Anomaly Detection

Feature-Aligned Stacked Autoencoder: A Novel Semisupervised Deep Learning Model for Pattern Classification of Industrial Faults

Class-Aware Contrastive Optimization for Imbalanced Text Classification

Diversified feature representation via deep auto-encoder ensemble through multiple activation functions

Hyperspectral Remote Sensing Image Classification Based on Auto-Encoder

Feature Selection for Imbalanced Data with Deep Sparse Autoencoders Ensemble

A New Modal Autoencoder for Functionally Independent Feature Extraction

Dual Balanced Class-Incremental Learning With im-Softmax and Angular Rectification

Dual Compensation Residual Networks for Class Imbalanced Learning

Boarding for ISS: Imbalanced Self-Supervised: Discovery of a Scaled Autoencoder for Mixed Tabular Datasets

An informative dual ForkNet for video anomaly detection