Abstract:Incremental learning is important for memory critical systems, especially when the growth of information technology has pushed the memory and storage costs to limits. Despite the great amount of effort researching into incremental classification paradigms and algorithms, the regression is given far less attention. In this paper, an incremental regression framework that is able to model the linear and nonlinear relationships between response variables and explanatory variables is proposed. A three layer feed-forward neural network structure is devised where the weights of the hidden layer are trained by topology learning neural networks. A Gaussian mixture weighted integrator is used to synthesize from the output of the hidden layer to give smoothed predictions. Two hidden layer parameters learning strategies whether by Growing Neural Gas (GNG) or the single layered Self-Organizing Incremental Neural Network (SOINN) are explored. The GNG strategy is more robust and flexible, and single layered SOINN strategy is less sensitive to parameter settings. Experiments are carried out on an artificial dataset and 6 UCI datasets. The artificial dataset experiments show that the proposed method is able to give predictions more smoothed than K-nearest-neighbor (KNN) and the regression tree. Comparing to the parametric method Support Vector Regression (SVR), the proposed method has significant advantage when learning on data with multi-models. Incremental methods including Passive and Aggressive regression, Online Sequential Extreme Learning Machine, Self-Organizing Maps and Incremental K-means are compared with the proposed method on the UCI datasets, and the results show that the proposed method outperforms them on most datasets.

Incremental updating method for big data feature learning

An Incremental Updating Method for Support Vector Machines

Incremental Clustering Algorithm of Neural Network

An Incremental Bayesian Network Structure Learning Algorithm Based on Local Updating Strategy

ICFD: an Incremental Learning Method Based on Data Feature Distribution.

Contrastive-based Bi-update Class Incremental Model and Its Application in Fault Diagnosis

Incremental Learning Without Looking Back: a Neural Connection Relocation Approach

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

A Survey of Research Work on Incremental Learning Based on Feedforward Neural Networks

Incremental learning with neural networks for computer vision: a survey

Adaptive knowledge transfer for class incremental learning

A Comparatively Research in Incremental Learning of Bayesian Networks

An Appraisal of Incremental Learning Methods

Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation

An Efficient Hybrid Incremental Algorithm for Complex-Valued Feedforward Neural Networks

Fast Evaluation of DNN for Past Dataset in Incremental Learning

Incremental Learning Using Conditional Adversarial Networks

Incremental Learning Bayesian Networks for Financial Data Modeling

An Adaptive Statistical Neural Network Model

Knowledge Aggregation Networks for Class Incremental Learning.

A Gaussian Mixture Framework for Incremental Nonparametric Regression with Topology Learning Neural Networks