Abstract:Class Incremental Learning (CIL) is a paradigm that excels in efficiently training models on an expanding set of classes, while preserving performance in classes learned earlier. In the context of smart city development, CIL plays a crucial role in addressing the need for Networking Systems of Artificial Intelligence (NSAI) to continuously adapt to ever-evolving data. In recent years, the incremental network structure widely adopted in CIL suffers from poor knowledge distillation and rapid model parameter growth. In this paper, we introduce a novel strategy named dynamic regeneration with task-adaptive distillation (DRTAD), which dynamically adapts to new tasks, and sustains good performance even as the class set continues to grow. DRTAD adopts a two-stage training strategy: dynamic regeneration and dynamic retention. During dynamic regeneration, DRTAD enhances feature representation by creating a new feature extraction module that extracts features from new classes, while also utilizing features from previously learned classes. Additionally, DRTAD introduces task-adaptive distillation to improve the poor knowledge distillation, further mitigating catastrophic forgetting. During the dynamic retention phase, DRTAD achieves a higher pruning rate through RM operation. Comprehensive experiments on CIFAR-100 and ImageNet-100 datasets demonstrate DRTAD’s superior performance compared to existing CIL methods. Notably, in the CIFAR100-B50 5 steps incremental setting, DRTAD increases the last-phase accuracy from 65.51% to 70.54% (＋5.03%), while maintaining fewer parameters (−11.1%). Similarly, in the ImageNet100-B50 10 steps setting, the last-phase accuracy rises from 70.04% to 72.50% (＋2.46%). These results indicate DRTAD’s efficacy in mitigating catastrophic forgetting in incremental structure.

Efficient Incremental Learning Using Dynamic Correction Vector

Learning Robust Features with Incremental Auto-Encoders

An Incremental Learning Algorithm Based on Support Vector Domain Classifier

An Incremental Updating Method for Support Vector Machines

Incremental learning algorithm based on support vector machine

Maintaining Discrimination and Fairness in Class Incremental Learning

Adaptive knowledge transfer for class incremental learning

An Efficient Strategy for Catastrophic Forgetting Reduction in Incremental Learning

Efficient Incremental Training for Deep Convolutional Neural Networks

Incremental Learning for Online Data Using QR Factorization on Convolutional Neural Networks

Incremental Learning of Object Detectors without Catastrophic Forgetting

Contrastive-based Bi-update Class Incremental Model and Its Application in Fault Diagnosis

Incremental Learning Via Rate Reduction

Model Behavior Preserving for Class-Incremental Learning

Class Incremental Learning Via Dynamic Regeneration with Task-Adaptive Distillation

Incremental learning with neural networks for computer vision: a survey

Class Incremental Learning with Deep Contrastive Learning and Attention Distillation

Incremental Scene Classification Using Dual Knowledge Distillation and Classifier Discrepancy on Natural and Remote Sensing Images

Incremental Classifier Learning with Generative Adversarial Networks

Class incremental learning with probability dampening and cascaded gated classifier

Incremental class learning using variational autoencoders with similarity learning