Abstract:Highlights•The stochastic gradient support vector machine algorithm is proposed based on the negative gradient search and the recursive identification methods. •Using the multi-innovation identification theory as the optimization strategy to optimize the stochastic gradient support vector machine algorithm, a multi-innovation stochastic gradient support vector machine algorithm is proposed.•In order to improve the performance of the proposed algorithms, introducing the forgetting factor leads to the corresponding forgetting factor recursive algorithms.For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.

Gradient-based Parameter Estimation for a Nonlinear Exponential Autoregressive Time-series Model by Using the Multi-innovation

Data Filtering-Based Multi-innovation Stochastic Gradient Algorithm for Nonlinear Output Error Autoregressive Systems

Cauchy kernel correntropy‐based robust multi‐innovation identification method for the nonlinear exponential autoregressive model in non‐Gaussian environment

Additional fractional gradient descent identification algorithm based on multi-innovation principle for autoregressive exogenous models

Three‐stage multi‐innovation parameter estimation for an exponential autoregressive time‐series model with moving average noise by using the data filtering technique

Weighted multi‐innovation parameter estimation for a time‐varying Volterra–Hammerstein system with colored noise

Two-stage Gradient-based Iterative Estimation Methods for Controlled Autoregressive Systems Using the Measurement Data

Multi-innovation stochastic gradient identification method

Maximum likelihood based multi‐innovation stochastic gradient identification algorithms for bilinear stochastic systems with ARMA noise

Variable Smoothing Parameter of the Double Exponential Smoothing Forecasting Model and Its Application

Improve recursive identification using multi-iteration

Adaptive Multi-Innovation Gradient Identification Algorithms for a Controlled Autoregressive Autoregressive Moving Average Model

Two‐stage auxiliary model gradient‐based iterative algorithm for the input nonlinear controlled autoregressive system with variable‐gain nonlinearity

Multi‐innovation gradient‐based iterative identification methods for feedback nonlinear systems by using the decomposition technique

Multivariable CAR-like System Identification with Multi-innovation Gradient and Least Squares Algorithms

Joint two‐stage multi‐innovation recursive least squares parameter and fractional‐order estimation algorithm for the fractional‐order input nonlinear output‐error autoregressive model

Adaptive Accelerated Proximal Gradient Algorithm for Auto-Regressive Exogenous Models with Outliers

Identifiability Implies Robust, Globally Exponentially Convergent On-line Parameter Estimation: Application to Model Reference Adaptive Control

Multi-innovation identification method for time-varying systems

The Nesterov accelerated gradient algorithm for Auto-Regressive Exogenous models with random lost measurements: interpolation method and auxiliary model method

A novel multi-innovation gradient support vector machine regression method