Abstract:Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta learning-based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective.

A Survey of Research Work on Incremental Learning Based on Feedforward Neural Networks

A Novel Approach to Incremental SVM Learning Algorithm

Incremental learning with neural networks for computer vision: a survey

A Comparatively Research in Incremental Learning of Bayesian Networks

An Appraisal of Incremental Learning Methods

Incremental updating method for big data feature learning

Incremental Learning Without Looking Back: a Neural Connection Relocation Approach

Class-Incremental Learning: Survey and Performance Evaluation on Image Classification

Preliminary Exploration of Data Incremental Learning Method

A Gaussian Mixture Framework for Incremental Nonparametric Regression with Topology Learning Neural Networks

An Ensemble of Neural Networks Based Incremental Learning Algorithm with Weights-updated Voting

Incremental Clustering Algorithm of Neural Network

On the Exploration of Incremental Learning for Fine-grained Image Retrieval

A Broad Neural Network Structure for Class Incremental Learning.

Knowledge Aggregation Networks for Class Incremental Learning.

A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration

Incremental Learning with Concept Drift: A Knowledge Transfer Perspective.

A survey on few-shot class-incremental learning

Increment learning algorithm for structural covering method

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

Incremental Sequential Learning for Fuzzy Neural Networks