Abstract:A new adaptive sparse least squares support vector regression algorithm, referred to as SLSSVR has been introduced for the adaptive modeling of nonstationary systems. Using a sliding window of recent data set of size N to track t he non-stationary characteristics of the incoming data, our adaptive model is initially formulated based on least squares support vector regression with forgetting factor (without bias term). In order to obtain a sparse model in which some parameters are exactly zeros, a l 1 penalty was applied in parameter estimation in the dual problem. Furthermore we exploit the fact that since the associated system/kernel matrix in positive definite, the dual solution of least squares support vector machine without bias term, can be solved iteratively with guaranteed convergence. Furthermore since the models between two consecutive time steps there are (N-1) shared kernels/parameters, the online solution can be obtained efficiently using coordinate descent algorithm in the form of Gauss-Seidel algorithm with minimal number of iterations. This allows a very sparse model per time step to be obtained very efficiently, avoiding expensive matrix inversion. The real stock market dataset and simulated examples have shown that the proposed approaches can lead to superior performances in comparison with the linear recursive least algorithm and a number of online non-linear approaches in terms of modelling performance and model size.

A Newton Algorithm for Nonlinear Regression

A Novel Nonlinear Dynamic System Modeling Approach Using Radial Basis Function Neural Networks

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Support Vector Regression Based Nonlinear Model Reference Adaptive Control

γ-C plane and robustness in static reservoir for nonlinear regression estimation

A fast algorithm for solving large scale nonlinear optimization problems using RNN

Recursive Finite Newton Algorithm for Support Vector Regression in the Primal

Deep Residual Learning for Nonlinear Regression

A Stochastic Newton Method for Nonlinear Equations

Stochastic Sub-Sampled Newton Method with Variance Reduction

Stochastic Regularized Newton Methods for Nonlinear Equations

Fast Newton Method for Sparse Logistic Regression

An Extended Newton-type Algorithm for ℓ_2-Regularized Sparse Logistic Regression and Its Efficiency for Classifying Large-scale Datasets

An Extended Newton-type Algorithm for $\ell_2$-Regularized Sparse Logistic Regression and Its Efficiency for Classifying Large-scale Datasets

Generalized Newton–Raphson Algorithm for High Dimensional LASSO Regression

A novel parametric-insensitive nonparallel support vector machine for regression

A Regularized Newton Method for Monotone Nonlinear Equations and Its Application

Newton Method for Nonlinear Dynamic Systems with Adaptive Time Stepping

Comparison of &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$\ell _{1}$ &Lt;/tex-Math></inline-formula>-norm SVR and Sparse Coding Algorithms for Linear Regression

A New Newton-like Method for Solving Nonlinear Equations

Sparse Least Squares Support Vector Regression for Nonstationary Systems.