Parallel-in-Time Solutions with Random Projection Neural Networks

Marta M. Betcke,Lisa Maria Kreusser,Davide Murari

2024-08-19

Abstract:This paper considers one of the fundamental parallel-in-time methods for the solution of ordinary differential equations, Parareal, and extends it by adopting a neural network as a coarse propagator. We provide a theoretical analysis of the convergence properties of the proposed algorithm and show its effectiveness for several examples, including Lorenz and Burgers' equations. In our numerical simulations, we further specialize the underpinning neural architecture to Random Projection Neural Networks (RPNNs), a 2-layer neural network where the first layer weights are drawn at random rather than optimized. This restriction substantially increases the efficiency of fitting RPNN's weights in comparison to a standard feedforward network without negatively impacting the accuracy, as demonstrated in the SIR system example.

Numerical Analysis,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the problem of parallel-in-time solving of ordinary differential equations (ODEs), particularly by introducing random projection neural networks (RPNNs) as a coarse propagator into the Parareal algorithm. Specifically, the researchers hope to utilize RPNNs to improve solving efficiency while maintaining the convergence of the Parareal algorithm. The specific problems this paper attempts to solve are as follows: 1. **Parallel-in-time solving**: Traditional time-stepping methods are usually serial, meaning the next step can only be computed after the current step is completed. The Parareal algorithm achieves time parallelization by combining fine and coarse time integrators, updating initial conditions in each iteration. This allows the originally serial process to be handled in parallel, thereby increasing solving speed. 2. **Neural networks as coarse propagators**: This paper further extends the Parareal algorithm by using random projection neural networks (RPNNs) as coarse propagators. RPNNs are a type of two-layer neural network where the weights of the first layer are randomly fixed and not optimized, significantly reducing training time while maintaining high accuracy. Compared to traditional fully connected neural networks, this method can significantly reduce computation time without sacrificing precision. 3. **Theoretical guarantees and experimental validation**: The paper provides theoretical analysis proving the convergence of the proposed hybrid method and validates it in practical applications. Numerical simulations demonstrate the effectiveness and efficiency improvements of RPNNs in various problems, such as the Lorenz system and the SIR model. In summary, this paper improves the Parareal algorithm by introducing RPNNs to achieve faster and more efficient parallel-in-time solving of ordinary differential equations, and it proves its effectiveness through theoretical analysis and experiments.

Parallel-in-Time Solutions with Random Projection Neural Networks

RandNet-Parareal: a time-parallel PDE solver using Random Neural Networks

Parareal with a physics-informed neural network as coarse propagator

Nearest Neighbors GParareal: Improving Scalability of Gaussian Processes for Parallel-in-Time Solvers

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Space-time parallel scaling of Parareal with a Fourier Neural Operator as coarse propagator

RandONets: Shallow Networks with Random Projections for Learning Linear and Nonlinear Operators

Local Randomized Neural Networks with Discontinuous Galerkin Methods for Partial Differential Equations

A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks

RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators

Random Projection in Deep Neural Networks

A Fast Second-Order Parareal Solver for Fractional Optimal Control Problems

Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs

Random Projection Neural Networks of Best Approximation: Convergence theory and practical applications

Randomized neural network with Petrov–Galerkin methods for solving linear and nonlinear partial differential equations

Adaptive Growing Randomized Neural Networks for Solving Partial Differential Equations

GParareal: A time-parallel ODE solver using Gaussian process emulation

Parallel Neural Networks in Golang

A Priori Estimation of the Approximation, Optimization and Generalization Errors of Random Neural Networks for Solving Partial Differential Equations

Parallel Physics-Informed Neural Networks Method with Regularization Strategies for the Forward-Inverse Problems of the Variable Coefficient Modified KdV Equation

Exponential Runge-Kutta Parareal for Non-Diffusive Equations