Abstract:Neural network models have achieved great success in numerous disciplines in recent years, including image segmentation, object identification, and natural language processing (NLP). Incremental learning in these fields focuses on training models in a continuous data stream. As time goes by, more new data becomes available, and old data may become unavailable owing to resource constraints such as storage. As a result, when new data is continually arriving, the performance of the neural network model on the old data sample sometimes decreases significantly, a phenomenon known as catastrophic forgetting. Many corresponding strategies have been proposed to mitigate the catastrophic forgetting of neural network models, which are based on parameter regularization, data replay, and parameter isolation. This paper proposes an incremental learning method based on data feature distribution (ICFD). The method uses Gaussian distribution to generate features from old data to train neural network models based on the phenomenon that feature vectors obey multi-dimensional Gaussian distribution in feature space. This method avoids storing a large number of original samples, and the generated old class features contain more sample information. This method combines data playback and parameter regularization in concrete implementation. The experimental results of ICFD on the CIFAR-100 demonstrate that when the incremental step is 5, the average incremental accuracy is increased by 10.4%. When the incremental step is 10, the average incremental accuracy is improved by 8.1%.

Incremental Fisher linear discriminant based on data denoising

Generalization Performance of Fisher Linear Discriminant Based on Markov Sampling

Integrated Fisher Linear Discriminants: an Empirical Study

Least Square Incremental Linear Discriminant Analysis

ICFD: an Incremental Learning Method Based on Data Feature Distribution.

A Discriminant Model for the Pattern Recognition of Linearly Independent Samples

Fisher Discriminant Analysis Based on Nonlinear Mapping

Denoising Fisher Training for Neural Implicit Samplers

Adaptive Quasiconformal Kernel Fisher Discriminant Analysis Via Weighted Maximum Margin Criterion

Regularized Fisher Linear Discriminant Through Two Threshold Variation Strategies for Imbalanced Problems

Iterative Learning of Fisher Linear Discriminants for Handwritten Digit Recognition.

ON OPTIMISING THRESHOLDS OF FISHER LINEAR DISCRIMINANT

An efficient renovation on kernel Fisher discriminant analysis and face recognition experiments

A Variant Fisher Discriminant Analysis Algorithm and Its Applicationto Fault Diagnosis.

Incremental Discriminant Analysis on Interval-Valued Parameters for Emitter Identification

Incremental Regularized Least Squares for Dimensionality Reduction of Large-Scale Data.

Approximately Optimal Domain Adaptation with Fisher's Linear Discriminant

Latent Fisher Discriminant Analysis

Interest Filter Vs. Interest Operator: Face Recognition Using Fisher Linear Discriminant Based on Interest Filter Representation.

Making Fisher Discriminant Analysis Scalable.

Fisher's Linear Discriminant Embedded Metric Learning