A Convergence Theorem for the Parareal Algorithm Revisited

Ernest Scheiber

2024-05-11

Abstract:The subject of the paper is to verify the convergence conditions for the parareal algorithm using Gander and Hairer's theorem . The analysis is conducted in the case where the coarse integrator is the Euler method and the high-accuracy integrator is an explicit Runge-Kutta type method.

Numerical Analysis

What problem does this paper attempt to address?

Convergence Analysis of the Parareal Algorithm with Nonuniform Fine Time Grid

Shu-Lin Wu,Tao Zhou

DOI: https://doi.org/10.1137/23m1592481

IF: 3.039

2024-10-11

SIAM Journal on Numerical Analysis

Abstract:SIAM Journal on Numerical Analysis, Volume 62, Issue 5, Page 2308-2330, October 2024. In this paper, we study the convergence properties of the parareal algorithm with uniform coarse time grid and arbitrarily distributed (nonuniform) fine time grid, which may be changed at each iteration. We employ the backward-Euler method as the coarse propagator and a general single-step time-integrator as the fine propagator. Specifically, we consider two implementations of the coarse grid correction: the standard time-stepping mode and the parallel mode via the so-called diagonalization technique. For both cases, we prove that under certain conditions of the stability function of the fine propagator, the convergence factor of the parareal algorithm is not larger than that of the associated algorithm with a uniform fine time grid. Furthermore, we show that when such conditions are not satisfied, one can indeed observe degenerations of the convergence rate. The model that is used for performing the analysis is the Dahlquist test equation with nonnegative parameter, and the numerical results indicate that the theoretical results hold for nonlinear ODEs and linear ODEs where the coefficient matrix has complex eigenvalues.

mathematics, applied
A Parareal algorithm without Coarse Propagator?

Martin J. Gander,Mario Ohlberger,Stephan Rave

2024-09-04

Abstract:The Parareal algorithm was invented in 2001 in order to parallelize the solution of evolution problems in the time direction. It is based on parallel fine time propagators called F and sequential coarse time propagators called G, which alternatingly solve the evolution problem and iteratively converge to the fine solution. The coarse propagator G is a very important component of Parareal, as one sees in the convergence analyses. We present here for the first time a Parareal algorithm without coarse propagator, and explain why this can work very well for parabolic problems. We give a new convergence proof for coarse propagators approximating in space, in contrast to the more classical coarse propagators which are approximations in time, and our proof also applies in the absence of the coarse propagator. We illustrate our theoretical results with numerical experiments, and also explain why this approach can not work for hyperbolic problems.

Numerical Analysis
Analysis of Two Parareal Algorithms for Time-Periodic Problems

Martin J. Gander,Yao-Lin Jiang,Bo Song,Hui Zhang

DOI: https://doi.org/10.1137/130909172

IF: 2.968

2013-01-01

SIAM Journal on Scientific Computing

Abstract:The parareal algorithm, which permits us to solve evolution problems in a time parallel fashion, has created a lot of attention over the past decade. The algorithm has its roots in the multiple shooting method for boundary value problems, which in the parareal algorithm is applied to initial value problems, with a particular coarse approximation of the Jacobian matrix. It is therefore of interest to formulate parareal-type algorithms for time-periodic problems, which also couple the end of the time interval with the beginning, and to analyze their performance in this context. We present and analyze two parareal algorithms for time-periodic problems: one with a periodic coarse problem and one with a nonperiodic coarse problem. An interesting advantage of the algorithm with the nonperiodic coarse problem is that no time-periodic problems need to be solved during the iteration, since on the time subdomains, the problems are not time-periodic either. We prove for both linear and nonlinear problems convergence of the new algorithms, with linear bounds on the convergence. We also extend these results to evolution partial differential equations using Fourier techniques. We illustrate our analysis with numerical experiments, both for model problems and the realistic application of a nonlinear cooled reverse-flow reactor system of partial differential equations.
Multi-step variant of the parareal algorithm: convergence analysis and numerics

Katia AIT-AMEUR,Yvon Maday

DOI: https://doi.org/10.1051/m2an/2024014

2024-03-06

ESAIM: Mathematical Modelling and Numerical Analysis

Abstract:In this paper, we consider the problem of accelerating the numerical simulation of time dependent problems involving a multi-step time scheme by the parareal algorithm. The parareal method is based on combining predictions made by a coarse and cheap propagator, with corrections computed with two propagators: the previous coarse and a precise and expensive one used in a parallel way over the time windows. A multi-step time scheme can potentially bring higher approximation orders than plain one-step methods but the initialisation of each time window needs to be appropriately chosen. Our main contribution is the design and analysis of an algorithm adapted to this type of discretisation without being too much intrusive in the coarse or fine propagators. At convergence, the parareal algorithm provides a solution that coincides with the solution of the fine solver. In the classical version of parareal, the local initial condition of each time window is corrected at every iteration. When the fine and/or coarse propagators is a multi-step time scheme, we need to choose a consistent approximation of the solutions involved in the initialisation of the fine solver at each time windows. Otherwise, the initialisation error will prevent the parareal algorithm to converge towards the solution with fine solver’s accuracy. In this paper, we develop a variant of the algorithm that overcome this obstacle. Thanks to this, the parareal algorithm is more coherent with the underlying time scheme and we recover the properties of the original version. We show both theoretically and numerically that the accuracy and convergence of the multi-step variant of parareal algorithm are preserved when we choose carefully the initialisation of each time window.

mathematics, applied
A Superlinear Convergence Estimate for the Parareal Schwarz Waveform Relaxation Algorithm

Martin J. Gander,Yao-Lin Jiang,Bo Song

DOI: https://doi.org/10.1137/18m1177226

IF: 2.968

2019-01-01

SIAM Journal on Scientific Computing

Abstract:The parareal Schwarz waveform relaxation algorithm is a new space-time parallel algorithm for the solution of evolution partial differential equations. It is based on a decomposition of the entire space-time domain both in space and in time into smaller space-time subdomains, and then computes by an iteration in parallel on all these small space-time subdomains a better and better approximation of the overall solution in space-time. The initial conditions in the space-time subdomains are updated using a parareal mechanism, while the boundary conditions are updated using Schwarz waveform relaxation techniques. A first precursor of this algorithm was presented 15 years ago, and while the method works well in practice, the convergence of the algorithm is not yet understood, and to analyze it is technically difficult. We present in this paper for the first time an accurate superlinear convergence estimate when the algorithm is applied to the heat equation. We illustrate our analysis with numerical experiments including cases not covered by the analysis, which opens up many further research directions.
Exponential Runge-Kutta Parareal for Non-Diffusive Equations

Tommaso Buvoli,Michael L. Minion

2023-11-07

Abstract:Parareal is a well-known parallel-in-time algorithm that combines a coarse and fine propagator within a parallel iteration. It allows for large-scale parallelism that leads to significantly reduced computational time compared to serial time-stepping methods. However, like many parallel-in-time methods it can fail to converge when applied to non-diffusive equations such as hyperbolic systems or dispersive nonlinear wave equations. This paper explores the use of exponential integrators within the Parareal iteration. Exponential integrators are particularly interesting candidates for Parareal because of their ability to resolve fast-moving waves, even at the large stepsizes used by coarse propagators. This work begins with an introduction to exponential Parareal integrators followed by several motivating numerical experiments involving the nonlinear Schrödinger equation. These experiments are then analyzed using linear analysis that approximates the stability and convergence properties of the exponential Parareal iteration on nonlinear problems. The paper concludes with two additional numerical experiments involving the dispersive Kadomtsev-Petviashvili equation and the hyperbolic Vlasov-Poisson equation. These experiments demonstrate that exponential Parareal methods offer improved time-to-solution compared to serial exponential integrators when solving certain non-diffusive equations.

Numerical Analysis
Parareal algorithm via Chebyshev-Gauss spectral collocation method

Quan Zhou,Yicheng Liu,Shu-Lin Wu

2023-04-20

Abstract:We present the Parareal-CG algorithm for time-dependent differential equations in this work. The algorithm is a parallel in time iteration algorithm utilizes Chebyshev-Gauss spectral collocation method for fine propagator F and backward Euler method for coarse propagator G. As far as we know, this is the first time that the spectral method used as the F propagator of the parareal algorithm. By constructing the stable function of the Chebyshev-Gauss spectral collocation method for the symmetric positive definite (SPD) problem, we find out that the Parareal-CG algorithm and the Parareal-TR algorithm, whose F propagator is chosen to be a trapezoidal ruler, converge similarly, i.e., the Parareal-CG algorithm converge as fast as Parareal-Euler algorithm with sufficient Chebyhsev-Gauss points in every coarse grid. Numerical examples including ordinary differential equations and time-dependent partial differential equations are given to illustrate the high efficiency and accuracy of the proposed algorithm.

Numerical Analysis
On the Convergence of the Accelerated Riccati Iteration Method.

Prasanthan Rajasingam,Jianhong Xu

DOI: https://doi.org/10.1080/00207179.2018.1533253

IF: 2.102

2018-01-01

International Journal of Control

Abstract:In this paper, we establish results fully addressing two open problems proposed recently by I. Ivanov, see Nonlinear Analysis 69 (2008) 4012-4024, with respect to the convergence of the accelerated Riccati iteration method for solving the continuous coupled algebraic Riccati equation, or CCARE for short. These results confirm several desirable features of that method, including the monotonicity and boundedness of the sequences it produces, its capability of determining whether the CCARE has a solution, the extremal solutions it computes under certain circumstances, and its faster convergence than the regular Riccati iteration method.
Analysis of the Parareal Algorithm for Linear Parametric Differential Equations

Ren-Hao Zhang,Yao-Lin Jiang,Jun Li,Bo Song

DOI: https://doi.org/10.1080/00207160.2022.2153225

2022-01-01

Abstract:This paper presents a parareal algorithm with parameterized propagators for linear parametric differential equations over a wide range of parameters. Through transforming the initial value problem into nonparametric ODEs based on Taylor series, we construct the general parameterized fine and coarse propagators for the parareal algorithm in each time subinterval. Furthermore, to accelerate the convergence of the algorithm, the coarse propagator based on the waveform relaxation method is proposed. By analysing the computational complexity of the WR propagator and the general coarse propagator, we find these two propagators are appropriate for the different situations. Finally, the convergence analysis of the parareal algorithm with these propagators is presented and our analysis is illustrated with two numerical experiments.
Parareal Algorithm Implementation and Simulation in Julia

Tyler M. Masthay,Saverio Perugini

DOI: https://doi.org/10.48550/arXiv.1706.08569

2018-12-14

Abstract:We present a full implementation of the parareal algorithm---an integration technique to solve differential equations in parallel---in the Julia programming language for a fully general, first-order, initial-value problem. We provide a brief overview of Julia---a concurrent programming language for scientific computing. Our implementation of the parareal algorithm accepts both coarse and fine integrators as functional arguments. We use Euler's method and another Runge-Kutta integration technique as the integrators in our experiments. We also present a simulation of the algorithm for purposes of pedagogy and as a tool for investigating the performance of the algorithm.

Mathematical Software,Distributed, Parallel, and Cluster Computing,Numerical Analysis
Convergence Analysis of the Parareal Algorithms for Stochastic Maxwell Equations Driven by Additive Noise

Liying Zhang,Qi Zhang

2024-01-01

Abstract:In this paper, we investigate the strong convergence analysis of parareal algorithms for stochastic Maxwell equations with the damping term driven by additive noise. The proposed parareal algorithms proceed as two-level temporal parallelizable integrators with the stochastic exponential integrator as the coarse propagator and both the exact solution integrator and the stochastic exponential integrator as the fine propagator. It is proved that the convergence order of the proposed algorithms linearly depends on the iteration number. Numerical experiments are performed to illustrate the convergence order of the algorithms for different choices of the iteration number, the damping coefficient and the scale of noise.
GParareal: A time-parallel ODE solver using Gaussian process emulation

Kamran Pentland,Massimiliano Tamborrino,T. J. Sullivan,James Buchanan,L. C. Appel

DOI: https://doi.org/10.48550/arXiv.2201.13418

2022-09-23

Abstract:Sequential numerical methods for integrating initial value problems (IVPs) can be prohibitively expensive when high numerical accuracy is required over the entire interval of integration. One remedy is to integrate in a parallel fashion, "predicting" the solution serially using a cheap (coarse) solver and "correcting" these values using an expensive (fine) solver that runs in parallel on a number of temporal subintervals. In this work, we propose a time-parallel algorithm (GParareal) that solves IVPs by modelling the correction term, i.e. the difference between fine and coarse solutions, using a Gaussian process emulator. This approach compares favourably with the classic parareal algorithm and we demonstrate, on a number of IVPs, that GParareal can converge in fewer iterations than parareal, leading to an increase in parallel speed-up. GParareal also manages to locate solutions to certain IVPs where parareal fails and has the additional advantage of being able to use archives of legacy solutions, e.g. solutions from prior runs of the IVP for different initial conditions, to further accelerate convergence of the method -- something that existing time-parallel methods do not do.

Numerical Analysis,Computation
On the convergence of linear and nonlinear Parareal methods for the Cahn–Hilliard equation

Gobinda Garai,Bankim C. Mandal

DOI: https://doi.org/10.1016/j.cnsns.2024.108014

IF: 4.186

2024-04-10

Communications in Nonlinear Science and Numerical Simulation

Abstract:This paper introduces, analyses, and implements efficient time parallel methods for solving the Cahn–Hilliard (CH) equation. Efficient numerical methods for the CH equation are crucial due to its wide range of applications. In particular, simulating the CH equation often requires long computational times to obtain the solution during the phase coarsening stage. Therefore, there is a need to accelerate the computations using parallel methods in the time dimension. We propose linear and nonlinear Parareal methods for the CH equation, depending on the choice of the fine approximation. The effectiveness of our approach is demonstrated through numerical experiments.

mathematics, applied, interdisciplinary applications,mechanics,physics, mathematical, fluids & plasmas
Convergence Analysis of a Parareal-in-time Algorithm for the Incompressible Non-Isothermal Flows

Zhen Miao,Yao-Lin Jiang,Yun-Bo Yang

DOI: https://doi.org/10.1080/00207160.2018.1498484

2018-01-01

Abstract:This paper presents and analyzes a parareal-in-time scheme for the incompressible non-isothermal Navier-Stokes equations with Boussinesq approximation. Standard finite element method is adopted for the spatial discretization.The proposed algorithm is proved to be unconditional stability. The convergence factor of iteration error for the velocity and temperature is given at time-continuous case. It theoretically demonstrates the superlinearly convergence of the parareal iteration combined with finite element method for incompressible non-isothermal flows. Finally, several numerical experiments that confirm feasibility and applicability of the algorithm perform well as expected.
Multigrid Interpretations of the Parareal Algorithm Leading to an Overlapping Variant and MGRIT

Martin J. Gander,Felix Kwok,Hui Zhang

DOI: https://doi.org/10.1007/s00791-018-0297-y

2018-01-01

Computing and Visualization in Science

Abstract:The parareal algorithm is by construction a two level method, and there are several ways to interpret the parareal algorithm to obtain multilevel versions. We first review the three main interpretations of the parareal algorithm as a two-level method, a direct one, one based on geometric multigrid and one based on algebraic multigrid. The algebraic multigrid interpretation leads to the MGRIT algorithm, when using instead of only an F-smoother, a so called $\textit{FCF}$-smoother. We show that this can be interpreted at the level of the parareal algorithm as generous overlap in time. This allows us to generalize the method to even larger overlap, corresponding in MGRIT to $F(\textit{CF})^{\nu }$-smoothing, $\nu \ge 1$, and we prove two new convergence results for such algorithms in the fully non-linear setting: convergence in a finite number of steps, becoming smaller when $\nu $ increases, and a general superlinear convergence estimate for this generalized version of MGRIT. We illustrate our results with numerical experiments, both for linear and non-linear systems of ordinary and partial differential equations. Our results show that overlap only sometimes leads to faster algorithms.
A unified analysis framework for iterative parallel-in-time algorithms

M. J. Gander,T. Lunet,D. Ruprecht,R. Speck

DOI: https://doi.org/10.1137/22M1487163

2023-04-28

Abstract:Parallel-in-time integration has been the focus of intensive research efforts over the past two decades due to the advent of massively parallel computer architectures and the scaling limits of purely spatial parallelization. Various iterative parallel-in-time (PinT) algorithms have been proposed, like Parareal, PFASST, MGRIT, and Space-Time Multi-Grid (STMG). These methods have been described using different notations, and the convergence estimates that are available are difficult to compare. We describe Parareal, PFASST, MGRIT and STMG for the Dahlquist model problem using a common notation and give precise convergence estimates using generating functions. This allows us, for the first time, to directly compare their convergence. We prove that all four methods eventually converge super-linearly, and also compare them numerically. The generating function framework provides further opportunities to explore and analyze existing and new methods.

Numerical Analysis,Computational Engineering, Finance, and Science
Convergence Analysis of a Method for Variational Inclusions

M. H. Rashid,J. H. Wang,C. Li

DOI: https://doi.org/10.1080/00036811.2011.618127

2011-01-01

Applicable Analysis

Abstract:Consider the following variational inclusion problem: where f is differentiable in a neighbourhood of a solution and g is differentiable at , and F is a set-valued mapping, and the method introduced in Jean-Alexis and Pietrus [C. Jean-Alexis and A. Pietrus, On the convergence of some methods for variational inclusions, Rev. R. Acad. Cien. serie A. Mat. 102(2) (2008), pp. 355–361] for solving this problem: where ∇f(x) denotes the Fréchet derivative of f at x and [x, y; g] the first-order divided difference of g on the points x and y. Local converge analysis are provided for the method under the weaker conditions than Jean-Alexis and Pietrus (2008). Moreover, if ∇f and the first-order divided difference of g are p-Hölder continuous at a solution, then we show that this method converges superlinearly. In particular, our results extend the corresponding ones Jean-Alexis and Pietrus (2008), and fix a gap in the proof in (Jean-Alexis and Pietrus (2008), Theorem 1).
Analysis of Two New Parareal Algorithms Based on the Dirichlet-Neumann/Neumann-Neumann Waveform Relaxation Method for the Heat Equation.

Song Bo,Jiang Yao-Lin,Wang Xiaolong

DOI: https://doi.org/10.1007/s11075-020-00949-y

IF: 2.37

2020-01-01

Numerical Algorithms

Abstract:The Dirichlet-Neumann and Neumann-Neumann waveform relaxation methods are nonoverlapping spatial domain decomposition methods to solve evolution problems, while the parareal algorithm is in time parallel fashion. Based on the combinations of these space and time parallel strategies, we present and analyze two parareal algorithms based on the Dirichlet-Neumann and the Neumann-Neumann waveform relaxation method for the heat equation by choosing Dirichlet-Neumann/Neumann-Neumann waveform relaxation as two new kinds of fine propagators instead of the classical fine propagator. Both new proposed algorithms could be viewed as a space-time parallel algorithm, which increases the parallelism both in space and in time. We derive for the heat equation the convergence results for both algorithms in one spatial dimension. We also illustrate our theoretical results with numerical experiments finally.
A Parareal exponential integrator finite element method for linear parabolic equations

Jianguo Huang,Yuejin Xu

2024-12-02

Abstract:In this paper, for solving a class of linear parabolic equations in rectangular domains, we have proposed an efficient Parareal exponential integrator finite element method. The proposed method first uses the finite element approximation with continuous multilinear rectangular basis function for spatial discretization, and then takes the Runge-Kutta approach accompanied with Parareal framework for time integration of the resulting semi-discrete system to produce parallel-in-time numerical solution. Under certain regularity assumptions, fully-discrete error estimates in $L^2$-norm are derived for the proposed schemes with random interpolation nodes. Moreover, a fast solver can be provided based on tensor product spectral decomposition and fast Fourier transform (FFT), since the mass and coefficient matrices of the proposed method can be simultaneously diagonalized with an orthogonal matrix. A series of numerical experiments in various dimensions are also presented to validate the theoretical results and demonstrate the excellent performance of the proposed method.

Numerical Analysis

A Convergence Theorem for the Parareal Algorithm Revisited

Convergence Analysis of the Parareal Algorithm with Nonuniform Fine Time Grid

A Parareal algorithm without Coarse Propagator?

Analysis of Two Parareal Algorithms for Time-Periodic Problems

Multi-step variant of the parareal algorithm: convergence analysis and numerics

A Superlinear Convergence Estimate for the Parareal Schwarz Waveform Relaxation Algorithm

Exponential Runge-Kutta Parareal for Non-Diffusive Equations

Parareal algorithm via Chebyshev-Gauss spectral collocation method

On the Convergence of the Accelerated Riccati Iteration Method.

Analysis of the Parareal Algorithm for Linear Parametric Differential Equations

Parareal Algorithm Implementation and Simulation in Julia

Convergence Analysis of the Parareal Algorithms for Stochastic Maxwell Equations Driven by Additive Noise

GParareal: A time-parallel ODE solver using Gaussian process emulation

On the convergence of linear and nonlinear Parareal methods for the Cahn–Hilliard equation

Convergence Analysis of a Parareal-in-time Algorithm for the Incompressible Non-Isothermal Flows

Multigrid Interpretations of the Parareal Algorithm Leading to an Overlapping Variant and MGRIT

A unified analysis framework for iterative parallel-in-time algorithms

Convergence Analysis of a Method for Variational Inclusions

Analysis of Two New Parareal Algorithms Based on the Dirichlet-Neumann/Neumann-Neumann Waveform Relaxation Method for the Heat Equation.

A Parareal exponential integrator finite element method for linear parabolic equations