Abstract:With the promise of federated learning (FL) to allow for geographically-distributed and highly personalized services, the efficient exchange of model updates between clients and servers becomes crucial. FL, though decentralized, often faces communication bottlenecks, especially in resource-constrained scenarios. Existing data compression techniques like gradient sparsification, quantization, and pruning offer some solutions, but may compromise model performance or necessitate expensive retraining. In this paper, we introduce FedSZ, a specialized lossy-compression algorithm designed to minimize the size of client model updates in FL. FedSZ incorporates a comprehensive compression pipeline featuring data partitioning, lossy and lossless compression of model parameters and metadata, and serialization. We evaluate FedSZ using a suite of error-bounded lossy compressors, ultimately finding SZ2 to be the most effective across various model architectures and datasets including AlexNet, MobileNetV2, ResNet50, CIFAR-10, Caltech101, and Fashion-MNIST. Our study reveals that a relative error bound 1E-2 achieves an optimal tradeoff, compressing model states between 5.55-12.61x while maintaining inference accuracy within <0.5% of uncompressed results. Additionally, the runtime overhead of FedSZ is <4.7% or between of the wall-clock communication-round time, a worthwhile trade-off for reducing network transfer times by an order of magnitude for networks bandwidths <500Mbps. Intriguingly, we also find that the error introduced by FedSZ could potentially serve as a source of differentially private noise, opening up new avenues for privacy-preserving FL.

FedSQ: Sparse-Quantized Federated Learning for Communication Efficiency

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

Efficient asynchronous federated learning with sparsification and quantization

Communication-Efficient Federated Learning via Quantized Compressed Sensing

FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantization

FedSVRG Based Communication Efficient Scheme for Federated Learning in MEC Networks

Two-layer accumulated quantized compression for communication-efficient federated learning: TLAQC

FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization

FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications

Communication-efficient Federated Learning Via Quantized Clipped SGD

FedVQCS: Federated Learning via Vector Quantized Compressed Sensing

AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy

Communication-Efficient Design for Quantized Decentralized Federated Learning

Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning

Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes

Communication Efficient Federated Learning with Adaptive Quantization

FedFQ: Federated Learning with Fine-Grained Quantization

Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networks