Abstract:SUMMARY For software to fully exploit the computing power of emerging heterogeneous computers, not only must the required computational kernels be optimized for the specific hardware architectures but also an effective scheduling scheme is needed to utilize the available heterogeneous computational units and to hide the communication between them. As a case study, we develop a static scheduling scheme for the tridiagonalization of a symmetric dense matrix on multicore CPUs with multiple graphics processing units (GPUs) on a single compute node. We then parallelize and optimize the Basic Linear Algebra Subroutines (BLAS)‐2 symmetric matrix‐vector multiplication, and the BLAS‐3 low rank symmetric matrix updates on the GPUs. We demonstrate the good scalability of these multi‐GPU BLAS kernels and the effectiveness of our scheduling scheme on twelve Intel Xeon processors and three NVIDIA GPUs. We then integrate our hybrid CPU‐GPU kernel into computational kernels at higher‐levels of software stacks, that is, a shared‐memory dense eigensolver and a distributed‐memory sparse eigensolver. Our experimental results show that our kernels greatly improve the performance of these higher‐level kernels, not only reducing the solution time but also enabling the solution of larger‐scale problems. Because such symmetric eigenvalue problems arise in many scientific and engineering simulations, our kernels could potentially lead to new scientific discoveries. Furthermore, these dense linear algebra algorithms present algorithmic characteristics that can be found in other algorithms. Hence, they are not only important computational kernels on their own but also useful testbeds to study the performance of the emerging computers and the effects of the various optimization techniques. Copyright © 2013 John Wiley & Sons, Ltd.

Model Guided Algorithm Optimization for Tridiagonal Solver on Many-Core Architectures

A Novel Compute-Efficient Tridiagonal Solver for Many-Core Architectures

Parallel Tridiagonal Solver on Sunway Many-Core Processors*

A Parallel Iterative Method for Solving Periodical Block-Tridiagonal Linear Equations.

A Parallel Function Evaluation Approach for Solution to Large-Scale Equation-Oriented Models.

A Hybrid Parallel Tridiagonal Solver on Multi-core Architectures

Parallel optimization and application of unstructured sparse triangular solver on new generation of Sunway architecture

A Hierarchical Tridiagonal System Solver for Heterogenous Supercomputers

A Fast Tridiagonal Solver for Intel MIC Architecture

Characterize and Optimize Dense Linear Solver on Multi-core CPUs

Parallel Computation Method For Solving Large Scale Equation-Oriented Models

Toward Efficient Structured-Grid Triangular Solver on Sunway Many-Core Processors

A Parallel Algorithm Based on Galerkin Theory for Block-Tridiagonal Linear Systems

swPTS: an efficient parallel Thomas split algorithm for tridiagonal systems on Sunway manycore processors

Performance Modeling and Optimization of Parallel LU-SGS on Many-Core Processors for 3D High-Order CFD Simulations

Multi-GPU Hybrid Programming Accelerated Three-Dimensional Phase-Field Model in Binary Alloy

A Fast Sparse Triangular Solver for Structured-grid Problems on Sunway Many-core Processor SW26010

Tridiagonalization of a dense symmetric matrix on multiple GPUs and its application to symmetric eigenvalue problems

A Parallel Algorithm for Near Tridiagonal Toeplitz Equations on Distributed-Memory Multicomputers

A Parallel Algorithm for Solving Triangular Equations under Distributed Memory Environments

Research on Linear Algebra Algorithm on Multi-core and Many-core Architecture Platform