Abstract:This paperproposes arecursively adapted radial basis function networkandprovides additional insights into several well-known techniques suchasradial basis function networks, resource allocating networks andstochastic gradient descent inreproducing kernel Hilbert spaces. Throughthis perspective, resource allocating networks areinvestigated ina moreprincipled way so thatissues ofconvergence and generalization canbemathematically analyzed intheleast meansquare framework. I.INTRODUCTION RESOURCE allocating networks (RAN)areanintuitive, veryappealing ideaintroduced byPlatt (1)butits convergence andgeneralization havenotbeenfully addressed mathematically. Recently, Kivinen, SmolaandWilliamson (2)introduced anonline kernel learning method(OKL)whichcanbe simply viewedasa stochastic gradient descent method applied ontheregularized costfunction ofaradial basis function network (RBFN). Interestingly, OKL bears much resemblance withRAN intermsoflearning strategy and network architecture. However, there isonefundamental difference. OKL employs aforgetting factor initstraining, whichisthelegacy oftheregularization termintheRBFN costfunction, toimpose stability inlearning andthus ensure generalization. Further statistical analysis concludes that this forgetting factor appears essential forits well-posedness (3). Therefore, this reasoning raises thequestion ifRAN iswell- posed. Generalization hasbeensubstantially studied inthe framework ofregularization theories (4), (5). Regularization asaremedy forill-posedness became widely knowndueto theworkofTikhonov (6). Insolving RBFN like problems, theTikhonov regularization isessentially a trade-off between fitting training dataandreducing solution norms. Consequently itreduces thesensitivity ofthesolution to small changes inthetraining dataandimposes stability on theill-posed problem. Ourworkimproves onrelated research inseveral ways: 1.We propose arecursively adapted RBFNandprovide the convergence conditions.

Gradient Learning Dynamics of Radial Basis Function Networks

Application of Gradient Radial Basis Function Network in the Modeling of MEMS Gyro's Random Drift

Natural Gradient Learning Algorithms for RBF Networks

Dynamics of learning near singularities in radial basis function networks.

Online Learning Dynamics of Radial Basis Function Neural Networks Near the Singularity

An improved conjugate gradient algorithm for radial basis function (RBF) networks modelling

Approximation of Function by Adaptively Growing Radial Basis Function Neural Networks.

Deterministic Annealing Learning of the Radial Basis Function Nets for Improving the Regression Ability of RBF Network

A Novel Radial Basis Function Neural Network For Approximation

The Growing Radial Basis Function (RBF) Neural Network and Its Applications.

A RBF Neural Network with Optimum Learning Rates and Its Application

From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression

Constructing radial basis function networks based on partial least squares modeling method

A two-level learning hierarchy for the radial basis function networks

The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models

Gradient Descent-Barzilai Borwein-Based Neural Network Tracking Control for Nonlinear Systems with Unknown Dynamics.

A distributed cooperative learning algorithm based on Zero-Gradient-Sum strategy using Radial Basis Function Network

Recursively AdaptedRadial Basis Function Networks Andits Relationship Toresource Allocating Networks Andonline Kernel Learning

Hybrid Learning Algorithm for RBF Neural Network

A Comparative Analysis of Optimization and Generalization Properties of Two-Layer Neural Network and Random Feature Models under Gradient Descent Dynamics

Some Practical Aspects On Incremental Training Of Rbf Network For Robot Behavior Learning