Abstract:Stochastic computing is a paradigm in which logical operations are performed on randomly generated bit streams. Complex arithmetic operations can be executed by simple logic circuits, resulting in a much smaller area footprint compared to conventional binary counterparts. However, the random or pseudorandom sources required for generating the bit streams are costly in terms of area and offset the advantages. Additionally, due to the inherent randomness, the computation lacks precision, limiting the applicability of this paradigm. Importantly, achieving reasonable accuracy in stochastic computing involves high latency. Recently, deterministic approaches to stochastic computing have been proposed, demonstrating that randomness is not a requirement. By structuring the computation deterministically, exact results can be obtained, and the latency greatly reduced. The bit stream generated adheres to a "unary" encoding, retaining the non-positional nature of the bits while discarding the random bit generation of traditional stochastic computing. This deterministic approach overcomes many drawbacks of stochastic computing, although the latency increases quadratically with each level of logic, becoming unmanageable beyond a few levels. In this paper, we present a method for approximating the results of the deterministic method while maintaining low latency at each level. This improvement comes at the cost of additional logic, but we demonstrate that the increase in area scales with the square root of n, where n represents the equivalent number of binary bits of precision. Our new approach is general, efficient, composable, and applicable to all arithmetic operations performed with stochastic logic. We show that this approach outperforms other stochastic designs for matrix multiplication (dot-product), which is an integral step in nearly all machine learning algorithms.

Coded Real Number Matrix Multiplication for On-Device Edge Computing

A Reconfigurable Multiplier for Signed Multiplications with Asymmetric Bit-Widths.

Coded Sparse Matrix Multiplication

On the Optimal Recovery Threshold of Coded Matrix Multiplication

Sparsity-Preserving Encodings for Straggler-Optimal Distributed Matrix Computations at the Edge

Distributed Matrix Multiplication with a Smaller Recovery Threshold through Modulo-based Approaches

Wireless Distributed Matrix-Vector Multiplication using Over-the-Air Computation and Analog Coding

A New Coding Scheme for Matrix-Vector Multiplication Via Universal Decodable Matrices.

Folded Polynomial Codes for Coded Distributed $AA^\top$-Type Matrix Multiplication

A Systematic Approach Towards Efficient Private Matrix Multiplication

Private and Secure Distributed Matrix Multiplication With Flexible Communication Load

Fully Private Grouped Matrix Multiplication with Colluding Workers

Distributed Matrix Computations with Low-weight Encodings

Coded Computation across Shared Heterogeneous Workers with Communication Delay

A Scalable Approach to Performing Multiplication and Matrix Dot-Products in Unary

Private Coded Computation for Machine Learning

Variable Coded Batch Matrix Multiplication

Decode-and-Compare: An Efficient Verification Scheme for Coded Distributed Edge Computing

Rateless Codes for Private Distributed Matrix-Matrix Multiplication

High Performance Matrix Multiplication on Many Cores