Abstract:Neural network models have achieved great success in numerous disciplines in recent years, including image segmentation, object identification, and natural language processing (NLP). Incremental learning in these fields focuses on training models in a continuous data stream. As time goes by, more new data becomes available, and old data may become unavailable owing to resource constraints such as storage. As a result, when new data is continually arriving, the performance of the neural network model on the old data sample sometimes decreases significantly, a phenomenon known as catastrophic forgetting. Many corresponding strategies have been proposed to mitigate the catastrophic forgetting of neural network models, which are based on parameter regularization, data replay, and parameter isolation. This paper proposes an incremental learning method based on data feature distribution (ICFD). The method uses Gaussian distribution to generate features from old data to train neural network models based on the phenomenon that feature vectors obey multi-dimensional Gaussian distribution in feature space. This method avoids storing a large number of original samples, and the generated old class features contain more sample information. This method combines data playback and parameter regularization in concrete implementation. The experimental results of ICFD on the CIFAR-100 demonstrate that when the incremental step is 5, the average incremental accuracy is increased by 10.4%. When the incremental step is 10, the average incremental accuracy is improved by 8.1%.

An Incremental Local Distribution Network for Unsupervised Learning.

A Self-Organizing Incremental Neural Network Based on Local Distribution Learning

An Incremental Network with Local Experts Ensemble.

Incremental Label Distribution Learning with Scalable Graph Convolutional Networks

An Incremental Neural Network for Online Supervised Learning and Topology Learning

ICFD: an Incremental Learning Method Based on Data Feature Distribution.

MDINet: Multidomain Incremental Network for Change Detection

An Incremental Network for On-Line Unsupervised Classification and Topology Learning.

Incremental Inductive Dynamic Network Community Detection.

Class Incremental Learning Based on Local Structure Constraints in Feature Space

An Incremental Learning Network Model Based on Random Sample Distribution Fitting

Incomplete Label Distribution Learning Based on Supervised Neighborhood Information

Incremental Embedding and Learning in the Local Discriminant Subspace with Application to Face Recognition

Deep Class Incremental Learning from Decentralized Data

Inaccurate Label Distribution Learning with Dependency Noise

A Self-Structurizing Neural Network for Online Incremental Learning

Instance-Dependent Inaccurate Label Distribution Learning.

A Structural Developmental Neural Network with Information Saturation for Continual Unsupervised Learning

A local distribution net for data clustering

Incremental Semi-Supervised Kernel Construction With Self-Organizing Incremental Neural Network And Application In Intrusion Detection

Learning to Classify With Incremental New Class