Abstract:Highlights•The stochastic gradient support vector machine algorithm is proposed based on the negative gradient search and the recursive identification methods. •Using the multi-innovation identification theory as the optimization strategy to optimize the stochastic gradient support vector machine algorithm, a multi-innovation stochastic gradient support vector machine algorithm is proposed.•In order to improve the performance of the proposed algorithms, introducing the forgetting factor leads to the corresponding forgetting factor recursive algorithms.For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.

A Gradient RBF Network Based on Multi-Innovation Algorithm for Nonstationary Time Series Prediction

A Seasonal Grbf Network for Nonstationary Time Series Prediction

Fast tunable gradient RBF networks for online modeling of nonlinear and nonstationary dynamic processes

A Nonlinear Adaptive Controller Based on RBF Networks

A New Optimization Algorithm for Non-Stationary Time Series Prediction Based on Recurrent Neural Networks.

Multi-grain Cascade Recurrent Neural Network for Nonlinear Time-varying Process Soft Sensor Modeling

Nonlinear time series modeling and prediction using RBF network with improved clustering algorithm

A novel multi-innovation gradient support vector machine regression method

Novel hybrid multi-head self-attention and multifractal algorithm for non-stationary time series prediction

Intelligent Optimization Prediction Algorithm for Statistical Data Based on Recurrent Neural Network

The Growing Radial Basis Function (RBF) Neural Network and Its Applications.

Additional fractional gradient descent identification algorithm based on multi-innovation principle for autoregressive exogenous models

Novel Modeling Method for Grey RBF Neural Network and Its Application

A Novel FRBF-Type Model for Nonlinear Time Series Prediction

A new hybrid recurrent artificial neural network for time series forecasting

Gradient-based Parameter Estimation for a Nonlinear Exponential Autoregressive Time-series Model by Using the Multi-innovation

Neural Differential Recurrent Neural Network with Adaptive Time Steps

Multi-innovation stochastic gradient identification method

Data Filtering-Based Multi-innovation Stochastic Gradient Algorithm for Nonlinear Output Error Autoregressive Systems

Self-growing RBF Neural Networks and Parameters Optimization

A Novel Algorithm for Global Optimization of RBF Neural Networks