Abstract:Federated learning (FL) has achieved great success as a privacy-preserving distributed training paradigm, where many edge devices collaboratively train a machine learning model by sharing the model updates instead of the raw data with a server. However, the heterogeneous computational and communication resources of edge devices give rise to stragglers that significantly decelerate the training process. To mitigate this issue, we propose a novel FL framework named stochastic coded federated learning (SCFL) that leverages coded computing techniques. In SCFL, before the training process starts, each edge device uploads a privacy-preserving coded dataset to the server, which is generated by adding Gaussian noise to the projected local dataset. During training, the server computes gradients on the global coded dataset to compensate for the missing model updates of the straggling devices. We design a gradient aggregation scheme to ensure that the aggregated model update is an unbiased estimate of the desired global update. Moreover, this aggregation scheme enables periodical model averaging to improve the training efficiency. We characterize the tradeoff between the convergence performance and privacy guarantee of SCFL. In particular, a more noisy coded dataset provides stronger privacy protection for edge devices but results in learning performance degradation. We further develop a contract-based incentive mechanism to coordinate such a conflict. The simulation results show that SCFL learns a better model within the given time and achieves a better privacy-performance tradeoff than the baseline methods. In addition, the proposed incentive mechanism grants better training performance than the conventional Stackelberg game approach.

Supplementary File: Cooperative Gradient Coding for Semi-Decentralized Federated Learning

Supplementary File: Coded Cooperative Networks for Semi-Decentralized Federated Learning

Deep Learning Based Coded Over-the-Air Computation for Personalized Federated Learning

Secure Federated Learning with Model Compression.

Secure Federated Learning over Wireless Communication Networks with Model Compression

Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation

Decentralized Federated Learning under Communication Delays.

Gradient Coding in Decentralized Learning for Evading Stragglers

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Asynchronous Federated Learning over Wireless Communication Networks.

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design

Stochastic Coded Federated Learning with Convergence and Privacy Guarantees

Over-the-Air Decentralized Federated Learning

FedNC: A Secure and Efficient Federated Learning Method with Network Coding

FedMDC: Enabling Communication-Efficient Federated Learning over Packet Lossy Networks Via Multiple Description Coding

Enabling Communication-Efficient Federated Learning via Distributed Compressed Sensing.

Decentralized Federated Learning: Balancing Communication and Computing Costs

Toward Efficient Federated Learning in Multi-Channeled Mobile Edge Network with Layerd Gradient Compression

SVDFed: Enabling Communication-Efficient Federated Learning via Singular-Value-Decomposition.

Gradient-Congruity Guided Federated Sparse Training