An Asynchronous Algorithm to Reduce the Number of Data Exchanges

Zhuo Tian,Yifeng Chen,Lei Zhang
DOI: https://doi.org/10.1007/978-3-030-38961-1_15
2020-01-01
Abstract:Communication or data movement cost is significantly higher than computation cost in existing large-scale clusters, for clusters having long network latency. For high-frequency parallel iterative applications, performance bottleneck is the long network latency caused by frequent data exchange. This paper presents an asynchronous algorithm capable of reducing the number of data exchanges among processes of parallel iterative applications. The proposed algorithm has been tested on a stencil-based parallel computation and compared with a BSP implementation of the same application. The asynchronous algorithm can effectively reduce the number of data exchanges at the expense of higher computation overhead and larger message size, performance can be improved up to 2.8x.
What problem does this paper attempt to address?