Abstract:From a kernel-based modeling point of view, one can consider the given differential equations together with its initial or boundary conditions as prior knowledge and seek the solution by means of Least Squares Support Vector Machines (LSSVMs) whose parameters are adjusted to minimize an appropriate error function. In particular, here we introduce a LS-SVM based framework for learning the solutions of dynamical systems governed by Ordinary Differential Equations, Differential Algebraic Equations (DAEs) as well as Partial Differential Equations (PDEs) [1, 2, 3]. The problem is formulated as an optimization problem in the primal-dual setting. The approximate solution in the primal is expressed in terms of the feature map and is forced to satisfy the system dynamics, initial/boundary conditions using a constrained optimization problem. The optimal representation of the solution is then obtained in the dual. For the linear and nonlinear cases, the model parameters are obtained by solving a system of linear and nonlinear equations, respectively. The proposed model utilizes few training points to learn a closed form approximate solution. Furthermore, it does not require index reduction techniques and can directly be applied to learn the state trajectories of the underlying implicit systems. The experimental results on different ODE systems, DEA systems with index from 0 to 3, as well as PDEs are presented and compared with analytic solutions to confirm the validity and applicability of the proposed method. Similarly, analogous approaches based on the introduced LS-SVM framework can be used to address inverse problems where given the observational data one in particular aims at estimating the unknown parameters of the system [4, 5]. Here we present an extended formulation using LS-SVM core model to estimate the unknown constant or time-varying parameters of the given dynamical system described by ordinary (delay) differential equations. Finally, some possibilities and future directions for considering deeper architectures will be discussed.

Solving systems of linear equations via gradient systems with discontinuous righthand sides: application to LS-SVM

Support Vector Classifiers Via Gradient Systems with Discontinuous Righthand Sides

Using Svm To Model And Control Nonlinear Dynamical Systems

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Convergence of Newton'S Method for Systems of Equations with Constant Rank Derivatives

Supervised Descent Method for Solving Nonlinear Least Squares Problems in Computer Vision

Solving Systems of Phaseless Equations Via Riemannian Optimization with Optimal Sampling Complexity

An Efficient Modified Residual-Based Algorithm for Large Scale Symmetric Nonlinear Equations by Approximating Successive Iterated Gradients

LS-SVM based solutions to differential equations

On the equivalence between SOR-type methods for linear systems and discrete gradient methods for gradient systems

Numerical solutions to one dimensional linear Volterra-Fredholm integral equations based on LS-SVM model

Solving higher order nonlinear ordinary differential equations with least squares support vector machines

A semi-conjugate gradient method for solving unsymmetric positive definite linear systems

Successive Projection for Solving Systems of Nonlinear Equations/Inequalities

Nondegenerate Piecewise Linear Systems: A Finite Newton Algorithm and Applications in Machine Learning

A Multilevel Low-Rank Newton Method with Super-linear Convergence Rate and its Application to Non-convex Problems

Convergence Properties of an Iterative Method for Solving Symmetric Non-linear Equations

Learning solutions to partial differential equations using LS-SVM

Convergence Analysis of a Class of Nonsmooth Gradient Systems.

Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent