Abstract:Coded computing has emerged as a promising framework for tackling significant challenges in large-scale distributed computing, including the presence of slow, faulty, or compromised servers. In this approach, each worker node processes a combination of the data, rather than the raw data itself. The final result then is decoded from the collective outputs of the worker nodes. However, there is a significant gap between current coded computing approaches and the broader landscape of general distributed computing, particularly when it comes to machine learning workloads. To bridge this gap, we propose a novel foundation for coded computing, integrating the principles of learning theory, and developing a framework that seamlessly adapts with machine learning applications. In this framework, the objective is to find the encoder and decoder functions that minimize the loss function, defined as the mean squared error between the estimated and true values. Facilitating the search for the optimum decoding and functions, we show that the loss function can be upper-bounded by the summation of two terms: the generalization error of the decoding function and the training error of the encoding function. Focusing on the second-order Sobolev space, we then derive the optimal encoder and decoder. We show that in the proposed solution, the mean squared error of the estimation decays with the rate of $\mathcal{O}(S^3 N^{-3})$ and $\mathcal{O}(S^{\frac{8}{5}}N^{\frac{-3}{5}})$ in noiseless and noisy computation settings, respectively, where $N$ is the number of worker nodes with at most $S$ slow servers (stragglers). Finally, we evaluate the proposed scheme on inference tasks for various machine learning models and demonstrate that the proposed framework outperforms the state-of-the-art in terms of accuracy and rate of convergence.

Distance Function Learning in Error-Correcting Output Coding Framework

An Improved Error-Correcting Output Coding Framework with Kernel-Based Decoding

An Error-Resilient Coding Scheme For Still Color Image Wireless Transmission In Industrial Environment

A novel soft-coded error-correcting output codes algorithm

Optimized Weighted Decoding for Error-Correcting Output Codes

A self-adaptive soft-recoding strategy for performance improvement of error-correcting output codes

Weight optimization and layered clustering-based ECOC

The design of variable-length coding matrix for improving error correcting output codes

The design of dynamic ensemble selection strategy for the error-correcting output codes family

Error-Correcting Output Codes with Ensemble Diversity for Robust Learning in Neural Networks

Learning ECOC Code Matrix for Multiclass Classification with Application to Glaucoma Diagnosis

Linear Distance Coding for Image Classification

Robust Multi-View Face Detection Using Error Correcting Output Codes

Joint Error Concealment and Error Recovery for Consecutive Frame Losses under the Unbalanced Multiple Description Coding Architecture

Cross-layer iterative decoding of irregular LDPC codes using cyclic redundancy check codes

Coded Computing for Resilient Distributed Computing: A Learning-Theoretic Framework

A Genie-Aided Approach to Error Floor Estimation for Spatially Coupled Serially Concatenated Codes

The design of error-correcting output codes algorithm for the open-set recognition

Learning Linear Block Error Correction Codes

Deep N-ary Error Correcting Output Codes

Error-Correcting Codes in Projective Spaces via Rank-Metric Codes and Ferrers Diagrams