Abstract:Iterative applications such as stencil problem are the most important and time-consuming computing core for numerical simulations and scientific applications. Iterative applications often require high frequency iterations. The dependency computations between steps lead to frequent data exchanges. Existing large-scale clusters tend to have long memory access latency and network latency. Long latency results in higher communication or data movement costs than computating costs. Therefore, the main performance bottleneck of stencil algorithm on large-scale clusters is how to improve the iterative frequency. This paper proposes a new asynchronous algorithm which could reduce the number of data exchanges among processes of high-frequency iterative applications. The asynchronous algorithm consists of multiple rounds and communication only occurs after each round. The input data comes from the asynchronous communication, resulting in inaccurate but speculative computations. The speculative computations could improve accuracy and speed up convergence. In addition, the speculative computations can automatically correct the precision which is caused by asynchronous communication. The proposed algorithm has been tested on a stencil-based parallel computation and compared with a BSP implementation of the same application. The asynchronous algorithm can effectively reduce the number of data exchanges at the expense of higher computation overhead and larger message size, performance can be improved up to 2.8x.

Nonlinear Asynchronous Block Iterative Method

Asynchronous block-iterative methods for almost linear equations

Inexact Block Newton Methods for Solving Nonlinear Equations

Parallel Block-iterative Methods: Application to the Load Flow Problem

On the Convergence of Parallel Asynchronous Block-Iterative Computations

On a Nonlinear Fast Deterministic Block Kaczmarz Method for Solving Nonlinear Equations

Generalized multisplitting asynchronous iteration

A Parallel Function Evaluation Approach for Solution to Large-Scale Equation-Oriented Models.

Parallel Computation Method For Solving Large Scale Equation-Oriented Models

Scalable Hierarchical Parallel Algorithm for the Solution of Super Large-Scale Sparse Linear Equations

Asynchronous Algorithms for Solving Linear Programs

GPU-based Parallel Solver Via the Kantorovich Theorem for the Nonlinear Bernstein Polynomial Systems.

A class of pseudoinverse-free greedy block nonlinear Kaczmarz methods for nonlinear systems of equations

On a Fast Deterministic Block Kaczmarz Method for Solving Large-Scale Linear Systems.

The Existence the Solution of Nonlinear Discrete Schemes and Convergence of a Linearized Iterative Method for time-dependent PNP Equations

A Non-Krylov subspace Method for Solving Large and Sparse Linear System of Equations

Scalable Distributed Least Square Algorithms for Large-Scale Linear Equations Via an Optimization Approach

Collective Communication Optimization for Solving Linear Algebraic Equations

A Networked Parallel Algorithm for Solving Linear Algebraic Equations

An Asynchronous Algorithm for Optimizing the Communication Performance