Abstract:Straggler nodes are well-known bottlenecks of distributed matrix computations which induce reductions in computation/communication speeds. A common strategy for mitigating such stragglers is to incorporate Reed-Solomon based MDS (maximum distance separable) codes into the framework; this can achieve resilience against an optimal number of stragglers. However, these codes assign dense linear combinations of submatrices to the worker nodes. When the input matrices are sparse, these approaches increase the number of non-zero entries in the encoded matrices, which in turn adversely affects the worker computation time. In this work, we develop a distributed matrix computation approach where the assigned encoded submatrices are random linear combinations of a small number of submatrices. In addition to being well suited for sparse input matrices, our approach continues have the optimal straggler resilience in a certain range of problem parameters. Moreover, compared to recent sparse matrix computation approaches, the search for a "good" set of random coefficients to promote numerical stability in our method is much more computationally efficient. We show that our approach can efficiently utilize partial computations done by slower worker nodes in a heterogeneous system which can enhance the overall computation speed. Numerical experiments conducted through Amazon Web Services (AWS) demonstrate up to 30% reduction in per worker node computation time and 100x faster encoding compared to the available methods.

Straggler-tolerant stationary methods for linear systems

Developing iterative algorithms to solve Sylvester tensor equations

A general class of iterative splitting methods for solving linear systems

On the stability and performance of the solution of sparse linear systems by partitioned procedures

Preserving Sparsity and Privacy in Straggler-Resilient Distributed Matrix Computations

Distributed Matrix Computations with Low-weight Encodings

Stable Solutions of Linear Systems Involving Long Chain of Matrix Multiplications

Scalable Distributed Least Squares Algorithm for Linear Algebraic Equations via Scheduling

Virtual Transmission Method, A New Distributed Algorithm to Solve Sparse Linear Systems

A rational conjugate gradient method for linear ill-conditioned problems

A doubly stochastic block Gauss-Seidel algorithm for solving linear equations

Randomized Iterative Methods for Linear Systems

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

A Networked Parallel Algorithm for Solving Linear Algebraic Equations

Iterative Methods for Sparse Symmetric Multilinear Systems

A Novel Greedy Kaczmarz Method For Solving Consistent Linear Systems

Iterative methods for solving factorized linear systems

Empirical Study of Straggler Problem in Parameter Server on Iterative Convergent Distributed Machine Learning

Gradient-based iterative approach for solving constrained systems of linear matrix equations

Distributed matrix multiplication with straggler tolerance over very small field

Distributed matrix multiplication with straggler tolerance using algebraic function fields