Abstract:We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may also arise from spatial discretization of partial differential equations (PDEs). The internal weights are fixed to ones while the unknown weights between the hidden and output layer are computed with Newton's iterations using the Moore-Penrose pseudo-inverse for low to medium scale and sparse QR decomposition with L 2 regularization for medium- to large-scale systems. Building on previous works on random projections, we also prove its approximation accuracy. To deal with stiffness and sharp gradients, we propose an adaptive step-size scheme and address a continuation method for providing good initial guesses for Newton iterations. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled and the number of basis functions are "parsimoniously" chosen based on bias-variance trade-off decomposition. To assess the performance of the scheme in terms of both numerical approximation accuracy and computational cost, we used eight benchmark problems (three index-1 DAEs problems, and five stiff ODEs problems including the Hindmarsh-Rose neuronal model of chaotic dynamics and the Allen-Cahn phase-field PDE). The efficiency of the scheme was compared against two stiff ODEs/DAEs solvers, namely, ode15s and ode23t solvers of the MATLAB ODE suite as well as against deep learning as implemented in the DeepXDE library for scientific machine learning and physics-informed learning for the solution of the Lotka-Volterra ODEs included in the demos of the library. A software/toolbox in Matlab (that we call RanDiffNet) with demos is also provided.

On a Dynamic Variant of the Iteratively Regularized Gauss-Newton Method with Sequential Data.

A Stochastic Iteratively Regularized Gauss-Newton Method

Application of the iteratively regularized Gauss–Newton method to parameter identification problems in Computational Fluid Dynamics

Efficient generalized Golub-Kahan based methods for dynamic inverse problems

On the Convergence of A Data-Driven Regularized Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

Anderson Accelerated Gauss-Newton-guided deep learning for nonlinear inverse problems with Application to Electrical Impedance Tomography

Iterative regularization for ensemble data assimilation in reservoir models

Iterated INLA for State and Parameter Estimation in Nonlinear Dynamical Systems

Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs

Iterated Kalman Methodology For Inverse Problems

A Compound Gaussian Least Squares Algorithm and Unrolled Network for Linear Inverse Problems

Sequential bi-level regularized inversion with application to hidden reaction law discovery

Deep Regularized Compound Gaussian Network for Solving Linear Inverse Problems

Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows

Iterative Methods for Navier--Stokes Inverse Problems

An accelerated inexact Newton regularization scheme with a learned feature-selection rule for non-linear inverse problems

Filter Based Methods for Statistical Linear Inverse Problems

Multi-innovation-based online variable projection algorithm for a class of nonlinear models

Iterative methods for Navier-Stokes inverse problems

Solving Linear Inverse Problems Using Higher-Order Annealed Langevin Diffusion