Abstract:We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on $[-R_m,R_m]$ and fixed, where $R_m$ is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal value of $R_m$ based on the differential evolution algorithm. The presented method enables us to illuminate the characteristics of the optimal $R_m$ for two types of ELM configurations: (i) Single-Rm-ELM, in which a single $R_m$ is used for generating the random coefficients in all the hidden layers, and (ii) Multi-Rm-ELM, in which multiple $R_m$ constants are involved with each used for generating the random coefficients of a different hidden layer. We adopt the optimal $R_m$ from this method and also incorporate other improvements into the ELM implementation. In particular, here we compute all the differential operators involving the output fields of the last hidden layer by a forward-mode auto-differentiation, as opposed to the reverse-mode auto-differentiation in a previous work. These improvements significantly reduce the network training time and enhance the ELM performance. We systematically compare the computational performance of the current improved ELM with that of the finite element method (FEM), both the classical second-order FEM and the high-order FEM with Lagrange elements of higher degrees, for solving a number of linear and nonlinear PDEs. It is shown that the current improved ELM far outperforms the classical FEM. Its computational performance is comparable to that of the high-order FEM for smaller problem sizes, and for larger problem sizes the ELM markedly outperforms the high-order FEM.

Numerical solution for initial and boundary value problems of high-order ordinary differential equations using Hermite neural network algorithm with improved extreme learning machine

Numerical solution for high-order ordinary differential equations using H-ELM algorithm

Solving Partial Differential Equation Based on Bernstein Neural Network and Extreme Learning Machine Algorithm

Solving two-dimensional linear partial differential equations based on Chebyshev neural network with extreme learning machine algorithm

HNS: An efficient hermite neural solver for solving time-fractional partial differential equations

Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations

Radial basis function neural network with extreme learning machine algorithm for solving ordinary differential equations

Solving High-Dimensional Partial Differential Equations Using Tensor Neural Network and A Posteriori Error Estimators

Numerical Solution and Bifurcation Analysis of Nonlinear Partial Differential Equations with Extreme Learning Machines

Extreme Learning Machine with Kernels for Solving Elliptic Partial Differential Equations

Solving High Dimensional Partial Differential Equations Using Tensor Neural Network and A Posteriori Error Estimators

Solving higher order nonlinear ordinary differential equations with least squares support vector machines

Deep Neural Network Solutions for Oscillatory Fredholm Integral Equations

A piecewise neural network method for solving large interval solution to initial value problem of ordinary differential equations

Piecewise Neural Network Method for Solving Large Interval Solutions to Initial Value Problems of Ordinary Differential Equations

A new algorithm for solving nonlinear parabolic equations using extreme learning machine method with parameter retention

A Learning Based Numerical Method for Helmholtz Equations with High Frequency

A Normal Equation-Based Extreme Learning Machine for Solving Linear Partial Differential Equations

Numerical solution for high-dimensional partial differential equations based on deep learning with residual learning and data-driven learning

Legendre Neural Network Method for Several Classes of Singularly Perturbed Differential Equations Based on Mapping and Piecewise Optimization Technology

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements