Abstract:Large-scale supercomputers equipped with GPUs as accelerators are potential to satisfy the future Exascale computing. In this work the solution of large and sparse linear systems of equations by using the Krylov subspace methods, which is crucial for the overall performance of many industrial and scientific applications, is chosen to be accelerated by GPUs’ greatly enlarged computing power. To fulfill this objective on the target hardware with a large amount of heterogeneous computing nodes, two main contributions are included in this work. First we propose a communication avoiding variant of the BICGStab solution method which reduces the global synchronization points per iteration from 3 in the classical BICGStab method to 1 in the improved variant. The superiority in terms of a reduction of the expensive global communications via all computing processes can be expected on a large-scale distributed memory cluster. Second, to handle the host-to-accelerator data transfers, the main challenge encountered in the usage of heterogeneous architecture, a communication overlapped implementation of the sparse matrix–vector multiplication is proposed since this kernel features heavily in the Krylov subspace methods. Linear systems of equations arising from the incompressible Navier–Stokes equations are used to evaluate the proposed solution and optimization methods. Evaluations of the GPU and CPU implementations are conducted on up to 256 GPUs and 4096 CPU cores, respectively. It is revealed that to obtain the same computation time a two times reduction of the number of computing nodes is achieved by using the GPU implementation on the heterogeneous node equipped with 4 GPUs and a 32-core CPU. This result can be seen as the advantage of the heterogeneous architecture from the view point of applications, which motivates a wide utilization in other related areas.

Performance evaluation and analysis of sparse matrix and graph kernels on heterogeneous processors

Sparse matrix partitioning for optimizing SpMV on CPU-GPU heterogeneous platforms

Performance Evaluation of Hybrid Programming Patterns for Large CPU/GPU Heterogeneous Clusters.

GPU computing using concurrent kernels: A case study

High Performance Graph Analytics with Productivity on Hybrid CPU-GPU Platforms

Performance Optimization for CPU-GPU Heterogeneous Parallel System

A Hybrid Computing Method of Spmv on Cpu-Gpu Heterogeneous Computing Systems

A Hybrid CPU-GPU Multifrontal Optimizing Method in Sparse Cholesky Factorization

Heterogeneous systems for energy efficient scientific computing

Energy Cost Evaluation of Parallel Algorithms for Multiprocessor Systems

High Performance Computing Via a GPU

Understanding Co-Run Performance on CPU-GPU Integrated Processors: Observations, Insights, Directions

CPU Versus GPU: Which Can Perform Matrix Computation Faster—performance Comparison for Basic Linear Algebra Subprograms

Heterogeneous CPU+GPU-Accelerated FDTD for Scattering Problems with Dynamic Load Balancing

CPU GPU computing: Overview, optimization, and applications

Execution of Compound Multi-Kernel OpenCL Computations in Multi-CPU/Multi-GPU Environments

Efficient Matrix Factorization on Heterogeneous CPU-GPU Systems

HeteroCore GPU to Exploit TLP-Resource Diversity

Exploration on Task Scheduling Strategy for CPU-GPU Heterogeneous Computing System

GPU Computing Performance Analysis on Matrix Multiplication

Towards a Heterogeneous Architecture Solver for the Incompressible Navier–Stokes Equations