Abstract:Communication compression, a technique aiming to reduce the information volume to be transmitted over the air, has gained great interests in Federated Learning (FL) for the potential of alleviating its communication overhead. However, communication compression brings forth new challenges in FL due to the interplay of compression-incurred information distortion and inherent characteristics of FL such as partial participation and data heterogeneity. Despite the recent development, the performance of compressed FL approaches has not been fully exploited. The existing approaches either cannot accommodate arbitrary data heterogeneity or partial participation, or require stringent conditions on compression. In this paper, we revisit the seminal stochastic controlled averaging method by proposing an equivalent but more efficient/simplified formulation with halved uplink communication costs. Building upon this implementation, we propose two compressed FL algorithms, SCALLION and SCAFCOM, to support unbiased and biased compression, respectively. Both the proposed methods outperform the existing compressed FL methods in terms of communication and computation complexities. Moreover, SCALLION and SCAFCOM accommodates arbitrary data heterogeneity and do not make any additional assumptions on compression errors. Experiments show that SCALLION and SCAFCOM can match the performance of corresponding full-precision FL approaches with substantially reduced uplink communication, and outperform recent compressed FL methods under the same communication budget.

A Stochastic Gradient Approach for Communication Efficient Confederated Learning

Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach

TKAGFL: A Federated Communication Framework under Data Heterogeneity

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Confederated Learning: Federated Learning with Decentralized Edge Servers

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

Decentralized Federated Learning: Balancing Communication and Computing Costs

Adaptive Batchsize Selection and Gradient Compression for Wireless Federated Learning

Communication-Efficient Federated Learning: A Variance-Reduced Stochastic Approach with Adaptive Sparsification.

General Decentralized Federated Learning for Communication-Computation Tradeoff.

Asynchronous Federated Learning over Wireless Communication Networks.

Decentralized Federated Learning under Communication Delays.

Stochastic gradient compression for federated learning over wireless network

Supplementary File: Cooperative Gradient Coding for Semi-Decentralized Federated Learning

Stochastic Controlled Averaging for Federated Learning with Communication Compression

On the Convergence of Communication-Efficient Local SGD for Federated Learning

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication.

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing