Abstract:Matrix-matrix multiplication is a basic operation in linear algebra and an essential building block for a wide range of algorithms in various scientific fields. Theory and implementation for the dense, square matrix case are well-developed. If matrices are sparse, with application-specific sparsity patterns, the optimal implementation remains an open question. Here, we explore the performance of communication reducing 2.5D algorithms and one-sided MPI communication in the context of linear scaling electronic structure theory. In particular, we extend the DBCSR sparse matrix library, which is the basic building block for linear scaling electronic structure theory and low scaling correlated methods in CP2K. The library is specifically designed to efficiently perform block-sparse matrix-matrix multiplication of matrices with a relatively large occupation. Here, we compare the performance of the original implementation based on Cannon's algorithm and MPI point-to-point communication, with an implementation based on MPI one-sided communications (RMA), in both a 2D and a 2.5D approach. The 2.5D approach trades memory and auxiliary operations for reduced communication, which can lead to a speedup if communication is dominant. The 2.5D algorithm is somewhat easier to implement with one-sided communications. A detailed description of the implementation is provided, also for non ideal processor topologies, since this is important for actual applications. Given the importance of the precise sparsity pattern, and even the actual matrix data, which decides the effective fill-in upon multiplication, the tests are performed within the CP2K package with application benchmarks. Results show a substantial boost in performance for the RMA based 2.5D algorithm, up to 1.80x, which is observed to increase with the number of processes involved in the parallelization.

Exploring the Design Space of Distributed Parallel Sparse Matrix-Multiple Vector Multiplication

Automatic Tuning of Sparse Matrix-Vector Multiplication on Multicore Clusters.

Efficient Distributed-Memory Parallel Matrix-Vector Multiplication with Wide or Tall Unstructured Sparse Matrices

A sparsity-aware distributed-memory algorithm for sparse-sparse matrix multiplication

Efficient Sparse Matrix Kernels based on Adaptive Workload-Balancing and Parallel-Reduction

Distributed-Memory Parallel Algorithms for Sparse Matrix and Sparse Tall-and-Skinny Matrix Multiplication

Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs

SPMSD: an Partitioning-Strategy for Parallel General Sparse Matrix-Matrix Multiplication on GPU

Sparse Matrix-Vector Multiplication for Circuit Simulation

A novel data transformation and execution strategy for accelerating sparse matrix multiplication on GPUs.

Iterative Sparse Matrix-Vector Multiplication On In-Memory Cluster Computing Accelerated By Gpus For Big Data

A Work-Efficient Parallel Sparse Matrix-Sparse Vector Multiplication Algorithm

Optimization of Large-Scale Sparse Matrix-Vector Multiplication on Multi-GPU Systems

Sparse matrix partitioning for optimizing SpMV on CPU-GPU heterogeneous platforms

Node Aware Sparse Matrix-Vector Multiplication

On Parallelizing Matrix Multiplication by the Column-Row Method

Hypergraph Partitioning for Sparse Matrix-Matrix Multiplication

Performance Optimization for Parallel SpMV on a NUMA Architecture

Increasing the Efficiency of Sparse Matrix-Matrix Multiplication with a 2.5D Algorithm and One-Sided MPI

Optimizing sparse matrix-vector multiplication based on gpu

A Parallel Computing Method Using Blocked Format with Optimal Partitioning for SpMV on GPU