Incremental learning using RBF networks

Nong Sang,Kun Zeng

2004-01-01

Abstract:An incremental learning algorithm using RBF networks is proposed. When learning new patterns, old knowledge is reserved by retrieving interfered patterns. Simulated experiments proves the correctness and effectiveness of the algorithm.

What problem does this paper attempt to address?

Splicing System Based Genetic Algorithms for Developing Rbf Networks Models

Tao Jili,Wang Ning

DOI: https://doi.org/10.1016/s1004-9541(07)60065-2

IF: 3.8

2007-01-01

Chinese Journal of Chemical Engineering

Abstract:A splicing system based genetic algorithm is proposed to optimize dynamical radial basis function (RBF) neural network, which is used to extract valuable process information from input output data. The novel RBF network training technique includes the network structure into the set of function centers by compromising between the conflicting requirements of reducing prediction error and simultaneously decreasing model complexity. The effectiveness of the proposed method is illustrated through the development of dynamic models as a benchmark discrete example and a continuous stirred tank reactor by comparing with several different RBF network training methods.
Estimation of catalyst activity using improved RBFN

Bogao Liu,Jinshou Yu,Dao Huang,Xiangming Hua

1998-01-01

Abstract:An improved RBFN algorithm is proposed. Its training algorithm includes K-mean cluster algorithm and improved recursive method with discounted measurements. The former is used in on-line learning network centers, and the latter is used to adjust network weights. Simulation results about estimating the catalyst activity in an industrial fixed-bed reactor show that the training algorithm provides simple structure, good track performance, and fast learning speed.
A memristive RBF neural network and its application in unsupervised medical image segmentation

Sicong Liu,Chunbiao Li,Zhenyu Lu,Yongxin Li,Qiang Lai

DOI: https://doi.org/10.1140/epjs/s11734-022-00474-0

2022-03-07

The European Physical Journal Special Topics

Abstract:Recently, the image segmentation algorithm based on neural network has made great progress in the field of medical image segmentation, but it still faces many challenges such as the small set of training sample data, the lackness of background training data, weak network generalization ability, and poor network performance. To overcome the above difficulties, a new memristive Radial Basis Function (RBF) neural network is proposed for training effectively small sample set, which shows strong power of global searching and great generalization ability. Based on the memristive neural network and K-means clustering algorithm, a new algorithm for unsupervised image segmentation is designed. Through the analysis of experimental data, it is found that the algorithm proposed in this paper can segment medical images accurately without expert labeling data.
A Novel Approach to Incremental SVM Learning Algorithm

MAO Jian-yang,HUANG Dao

DOI: https://doi.org/10.3969/j.issn.1006-3080.2006.08.021

2006-01-01

Abstract:This paper presents a novel approach to incremental support vector machine (SVM) learning algorithm. It analyses the possible change of support vector set after new samples are added to training set. Based on the analysis result, a novel algorithm is presented. In this algorithm useless sample is discarded and knowledge is accumulated. The experiment result shows that this algorithm is more effective than traditional SVM while the classification precision is also guaranteed.
An Incremental Learning Algorithm Based on Support Vector Domain Classifier

Yinggang Zhao,Qinming He

DOI: https://doi.org/10.1109/coginf.2006.365593

2006-01-01

Abstract:Incremental learning technique is usually used to solve large-scale problem. We firstly gave a modified support vector machine (SVM) classification method - support vector domain classifier (SVDC), then an incremental learning algorithm based on SVDC was proposed. The basic idea of this incremental algorithm is to obtain the initial target concepts using SVDC during the training procedure and then update these target concepts by an updating model. Different from the existed incremental learning approaches, in our algorithm, the model updating procedure equals to solve a quadratic programming (QP) problem, and the updated model still owns the property of spars solution. Compared with other existed incremental learning algorithms, the inverse procedure of our algorithm (i.e. decreasing learning) is easy to conduct without extra computation. Experiment results show our algorithm is effective and feasible
AnIncremental Learning Algorithm Basedon Support Vector DomainClassifier

Yinggang Zhao,Qinming He

2006-01-01

Abstract:category (forexample, samples withcategory labelyi=1), yettheother partofsample's category isunknown, then we Incremental learning technique isusually usedtosolvecandesign newtypeofclassifier basedonSVDD named large-scale problem. Wefirstly gaveamodif edsupport vectorsupport vector domain classifier (SVDC).Thisnewclassifier machine (SVM)classification method--support vectoronlyneedtodescribe thedatawithknowncategory, then domainclassifer (SVDC), thenanincremental learning obtaining thedescription boundary ofthisclass ofdata. algorithm basedonSVDCwasproposed. Thebasic ideaof Finally, we canclassify theunknownbinary-class data thisincremental algorithm istoobtain theinitial targetaccording totheobtained boundary. concepts using SVDCduring thetraining procedure andthen Inthis paper ourincremental learning algorithm isbased update these target concepts byanupdating model. Difierent on SVDC, andthisalgorithm ismotivated by the fromtheexisted incremental learning approaches, inour person-learning procedure. Whenlearning a complicated algorithm, themodelupdating procedure equals tosolve a concept, people usually obtain ainitial concept byusing part quadratic programming (QP)problem, andtheupdated model ofuseful information, thenupdate theobtained concept by still ownstheproperty ofspars solution. Compared withotherutilizing new information. Intermofourincremental existed incremental learningalgorithms, theinverse procedure algorithm basedonSVDC,itfirstly utilize partofdata ofouralgorithm (i.e. decreasing learning) iseasytoconduct(memoryspace permitting), thenobtain aconcept (namely the without extra computation. Experiment results showour parameter ofobtained decision hypersurface) by SVDC algorithm iseffective andfeasible. learning algorithm, finally according totheinformation of decision hypersurface acquired inlaststep, update the parameter ofdecision hypersurface gained inlast step utilizing
A Nonlinear Adaptive Controller Based on RBF Networks

XH Chen,F Gao,JX Qian

DOI: https://doi.org/10.1109/icsmc.1996.569873

1996-01-01

Abstract:This paper proposes a nonlinear direct adaptive controller, based on radial basis function (RBF) networks. It is robust, reliable, efficient and simple. Compared with controllers based on BP networks, the proposed algorithm converges much more quickly without the problem of local minima. Simulation examples demonstrate the simplicity of the design procedure and the good characteristics of the control strategy. Moreover they illustrate that the controller possesses strong disturbance rejection and overcomes the drawback in outerpolation (accurate prediction outside the training domain) of neural network models.
Incremental learning algorithm based on support vector machine

HUANG Qi-chun,LIU Yang-guang,HE Qin-ming

DOI: https://doi.org/10.3785/j.issn.1008-973X.2008.12.015

2008-01-01

Abstract:An incremental learning algorithm based on support vector machine was proposed to process large scale data or data generated in batches. Initial goal concept learned by standard support vector machine algorithm was updated by an updating model. The model solved a convex quadratic programming similar to standard support vector machine algorithm. The algorithm proves that not only classification hyperplane but also the representation of classification hyperplane arenot changed during incremental learning procedure if the new samples arenot in the boundary in separable case. Decreasing learning procedure is easy to implement without extra computation and learning time is inversely proportional to incremental learning step compared with existing incremental learning support vector algorithms. Results show that the algorithm satisfies the three criteria of stability, improvement and recoverability.
An Incremental Updating Method for Support Vector Machines

Yangguang Liu,Qi Chen,Yongchuan Tang,Qinming He

DOI: https://doi.org/10.1007/978-3-540-24655-8_45

2004-01-01

Abstract:Support Vector Machines (SVMs) have become a popular tool for learning with large amounts of high dimensional data. But it may sometimes be preferable to learn incrementally from previous SVM results, as SVMs which involve the solution of a quadratic programming problem suffer from the problem of large memory requirement and CPU time when trained in batch mode on large data sets. And the SVMs may be used in online learning setting. In this paper an approach for incremental learning with Support Vector Machines is presented. We define the normal solution of the incremental learning for SVMs which is defined as the solution minimizing a given positive-definite quadratic form in the coordinates of the difference vector between the normal vectors at the (k-1)-th and k-th incremental step and discuss the relation to standard SVM. It was shown that concept learned at last step will not change if new data satisfy separable condition and empirical evidence is given to prove that this approach can effectively deal with changes in the target concept that are results of the incremental learning setting according to three evaluation criteria: stability, improvement and recoverability.
A Rbf Network Modeling Approach Combining Ga and Orthogonal Optimum Seeking Method

YJ Li,TJ Wu,MW Zhao

DOI: https://doi.org/10.1109/wcica.2000.863358

2000-01-01

Abstract:This paper proposes a novel approach for the training of RBF neural networks. It combines the genetic algorithm (GA) and orthogonal optimum seeking method, taking advantage of both the GA's global optimization ability and the orthogonal optimum seeking method that can quickly determine the numbers of the hidden nodes, to correctly determine the centers of the RBF network. Simulation results show that it is a simple method, and is more reliable and efficient than orthogonal optimum seeking method in training the RBF neural network for dynamic modeling
Convergence Analysis of an Incremental Approach to Online Inverse Reinforcement Learning

Zhuo-jun Jin,Hui Qian,Shen-yi Chen,Miao-liang Zhu

DOI: https://doi.org/10.1631/jzus.c1010010

2011-01-01

Abstract:Interest in inverse reinforcement learning （IRL） has recently increased,that is,interest in the problem of recovering the reward function underlying a Markov decision process （MDP） given the dynamics of the system and the behavior of an expert.This paper deals with an incremental approach to online IRL.First,the convergence property of the incremental method for the IRL problem was investigated,and the bounds of both the mistake number during the learning process and regret were provided by using a detailed proof.Then an online algorithm based on incremental error correcting was derived to deal with the IRL problem.The key idea is to add an increment to the current reward estimate each time an action mismatch occurs.This leads to an estimate that approaches a target optimal value.The proposed method was tested in a driving simulation experiment and found to be able to efficiently recover an adequate reward function.
Sparse Incremental Delta-Bar-Delta for System Identification

ying liu,yan ye,c g li

DOI: https://doi.org/10.4028/www.scientific.net/AMM.665.643

2014-01-01

Applied Mechanics and Materials

Abstract:Metalearning algorithm learns the base learning algorithm, targeted for improving the performance of the learning system. The incremental delta-bar-delta (IDBD) algorithm is such a metalearning algorithm. On the other hand, sparse algorithms are gaining popularity due to their good performance and wide applications. In this paper, we propose a sparse IDBD algorithm by taking the sparsity of the systems into account. The l(0) norm penalty is contained in the cost function of the standard IDBD, which is equivalent to adding a zero attractor in the iterations, thus can speed up convergence if the system of interest is indeed sparse. Simulations demonstrate that the proposed algorithm is superior to the competing algorithms in sparse system identification.
Self-tuning and approximation via RBF neural networks

Wenqing Ni,Shuicao Zheng,Guowei Lei,Linyang Xie

DOI: https://doi.org/10.1145/3580219.3580233

2023-01-28

Abstract:Radial Basis Function (RBF) is fit for real-time control task owing to its fast speed of learning by avoiding the local minimum problem. Moreover, RBF network can ensure the accuracy in control system owing to its robustness and adaptability. Up to now, the characters of RBF still need to be studied on how to improve the performance. In the paper, fuzzy RBF neural network and the combination with genetic algorithm are studied and simulated. In addition, RBF network is applied in self-tuning control system and validated

Engineering,Computer Science
Incremental Learning Bayesian Networks for the Stock Return Prediction

Shun Li,Da Shi,Bingwu Liu,Shaohua Tan

DOI: https://doi.org/10.1109/csae.2012.6272692

2012-01-01

Abstract:A group of hybrid incremental learning algorithms for Bayesian network structures is proposed in this paper. The center idea of these algorithms is combining the polynomial-time constraint-based technique and the search-and-score technique together to reduce the computational complexity. Our algorithms offer considerable computational complexity savings while obtaining better model accuracy compared to the existing incremental algorithms. One of these algorithms is also used to solve the stock return prediction problem which still has no good solutions till now.
A Two-Stage Learning Method to Configure Rbf Centers and Widths in Dynamic Environment Employing Immune Operations

Xiaofeng Ling,Xinbao Gong,Xiaogang Zang,Ronghong Jin

DOI: https://doi.org/10.14311/nnw.2011.21.021

2011-01-01

Neural Network World

Abstract:This paper proposes an immunity-based RBF training algorithm for nonlinear dynamic problems. Exploiting the locally-tuned structure of RBF network through immunological metaphor, a two-stage learning technique is proposed to configure RBF centers and widths in the hidden layer. Inspired by affinity maturation process of immune response, immune evolutionary mechanism (IEM) with memory operations is implemented in the learning stages to dynamically fine-tune the network performance. Experiment results also demonstrate that the algorithm has reached good performance with relatively low computational efforts in dynamic environments.
Using self-organizing incremental neural network (SOINN) For radial basis function networks

Jie Lu,Furao Shen,Jinxi Zhao

DOI: https://doi.org/10.1109/IJCNN.2014.6889649

2014-01-01

Abstract:This paper presents a batch learning algorithm and an online learning algorithm for radial basis function networks based on the self-organizing incremental neural network (SOINN), together referred to as SOINN-RBF. The batch SOINN-RBF is a combination of SOINN and least square algorithm. It achieves a comparable performance with SVM for regression. The online SOINN-RBF is based on the self-adaption procedure of SOINN and adopts the growing and pruning strategy of the minimal resource allocation network (MRAN). The growing and pruning criteria use the redefined significance, which is originally introduced by the growing and pruning algorithm for RBF (GGAP-RBF). Simulation results for both artificial and real-world data sets show that, comparing with other online algorithms, the online SOINN-RBF has comparable approximation accuracy, network compactness and better learning efficiency.
An Iterative Learning Algorithm Based on RBF Neural Network in Upper Limb Rehabilitation Robot

Zaixiang Pang,Tongyu Wang,Shuai Liu,Zhanli Wang,Xiyu Zhang,Yan Hao

DOI: https://doi.org/10.1109/ddcls52934.2021.9455640

2021-05-14

Abstract:Aiming at the non-linearity and uncertainty of patient spastic disturbance in the trajectory tracking control of upper limb rehabilitation robot, an iterative learning control algorithm is proposed based on RBF neural network. This paper considers repetitive nature of the rehabilitation robot system, the algorithm combines a single hidden layer feedforward neural network with iterative learning. In the upper limb rehabilitation process, the algorithm accelerate the convergence speed of the trajectory tracking error, and quickly suppress the interference in the interference environment. The Lyapunov stability theory is used to prove the globally asymptotic stability of the closed-loop system, then simulation proves the feasibility and effectiveness of the proposed algorithm.
An Integrated Hybrid Algorithm Based On Nature Inspired Evolutionary For Radial Basis Function Neural Network Learning

z y chen,r j kuo,t l hu

DOI: https://doi.org/10.1142/S0218213016500044

IF: 1.1

2016-01-01

International Journal on Artificial Intelligence Tools

Abstract:This paper intends to propose an integrated hybrid algorithm for training radial basis function neural network (RBFNN) learning. The proposed integrated of particle swarm and genetic algorithm based optimization (IPGO) algorithm is composed of two approaches based on particle swarm optimization (PSO) and genetic algorithm (GA) for gathering both their virtues to improve the learning performance of RBFNN. The diversity of individuals results in higher chance to search in the direction of global optimal instead of being confined to local optimal particularly in problem with higher complexity. The IPGO algorithm with PSO-based and GA-based approaches has shown promising results in some benchmark problems with three continuous test functions. After proposing the algorithm for these problems with result providing its outperforming performance, this paper supplements a practical application case for the papaya milk sales forecasting to expound the superiority of the IPGO algorithm. In addition, model evaluation results of the case have showed that the IPGO algorithm outperforms other algorithms and auto-regressive moving average (ARMA) models in terms of forecasting accuracy and execution time.
Robust Incremental Broad Learning System for Data Streams of Uncertain Scale

Linjun Zhong,C L Philip Chen,Jifeng Guo,Tong Zhang

DOI: https://doi.org/10.1109/TNNLS.2024.3396659

2024-05-17

Abstract:Due to its marvelous performance and remarkable scalability, a broad learning system (BLS) has aroused a wide range of attention. However, its incremental learning suffers from low accuracy and long training time, especially when dealing with unstable data streams, making it difficult to apply in real-world scenarios. To overcome these issues and enrich its relevant research, a robust incremental BLS (RI-BLS) is proposed. In this method, the proposed weight update strategy introduces two memory matrices to store the learned information, thus the computational procedure of ridge regression is decomposed, resulting in precomputed ridge regression. During incremental learning, RI-BLS updates two memory matrices and renews weights via precomputed ridge regression efficiently. In addition, this update strategy is theoretically analyzed in error, time complexity, and space complexity compared with existing incremental BLSs. Different from Greville's method used in the original incremental BLS, its results are closer to the solution of one-shot calculation. Compared with the existing incremental BLSs, the proposed method exhibits more stable time complexity and superior space complexity. The experiments prove that RI-BLS outperforms other incremental BLSs when handling both stable and unstable data streams. Furthermore, experiments demonstrate that the proposed weight update strategy applies to other random neural networks as well.

Incremental learning using RBF networks

Splicing System Based Genetic Algorithms for Developing Rbf Networks Models

Estimation of catalyst activity using improved RBFN

A memristive RBF neural network and its application in unsupervised medical image segmentation

A Novel Approach to Incremental SVM Learning Algorithm

An Incremental Learning Algorithm Based on Support Vector Domain Classifier

AnIncremental Learning Algorithm Basedon Support Vector DomainClassifier

A Nonlinear Adaptive Controller Based on RBF Networks

Incremental learning algorithm based on support vector machine

An Incremental Updating Method for Support Vector Machines

A Rbf Network Modeling Approach Combining Ga and Orthogonal Optimum Seeking Method

Convergence Analysis of an Incremental Approach to Online Inverse Reinforcement Learning

Sparse Incremental Delta-Bar-Delta for System Identification

Self-tuning and approximation via RBF neural networks

Incremental Learning Bayesian Networks for the Stock Return Prediction

A Two-Stage Learning Method to Configure Rbf Centers and Widths in Dynamic Environment Employing Immune Operations

Using self-organizing incremental neural network (SOINN) For radial basis function networks

An Iterative Learning Algorithm Based on RBF Neural Network in Upper Limb Rehabilitation Robot

An Integrated Hybrid Algorithm Based On Nature Inspired Evolutionary For Radial Basis Function Neural Network Learning

Robust Incremental Broad Learning System for Data Streams of Uncertain Scale