Abstract:Secure aggregation is a popular protocol in privacy-preserving federated learning, which allows model aggregation without revealing the individual models in the clear. On the other hand, conventional secure aggregation protocols incur a significant communication overhead, which can become a major bottleneck in real-world bandwidth-limited applications. Towards addressing this challenge, in this work we propose a lightweight gradient sparsification framework for secure aggregation, in which the server learns the aggregate of the sparsified local model updates from a large number of users, but without learning the individual parameters. Our theoretical analysis demonstrates that the proposed framework can significantly reduce the communication overhead of secure aggregation while ensuring comparable computational complexity. We further identify a trade-off between privacy and communication efficiency due to sparsification. Our experiments demonstrate that our framework reduces the communication overhead by up to 7.8x, while also speeding up the wall clock training time by 1.13x, when compared to conventional secure aggregation benchmarks.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is **reducing communication overhead in privacy - protected federated learning**. Specifically, although traditional secure aggregation protocols can protect users' privacy, they require each user to send the entire model to the server, which will bring significant communication burdens in large - scale networks, especially in bandwidth - limited wireless environments. To address this challenge, the paper proposes a lightweight gradient sparsification framework, named **SparseSecAgg**, which can send only part of the model parameters without revealing individual user model parameters, thereby significantly reducing communication overhead. ### Main Problems 1. **Communication Overhead Problem**: - **Limitations of Traditional Secure Aggregation**: Traditional secure aggregation protocols require each user to send complete model parameters, which will lead to huge communication overhead in large - scale networks, especially in bandwidth - limited wireless environments. - **Challenges of Gradient Sparsification**: Existing gradient sparsification techniques (such as random K - sparsification and Top - K sparsification) cannot be directly applied to secure aggregation because different users select different sparsification positions, resulting in pairwise masks that cannot be cancelled out, thus undermining security. 2. **Trade - off between Privacy and Communication Efficiency**: - **Privacy Protection**: Secure aggregation protocols protect users' privacy through pairwise additive masks, but sparsification may disrupt this masking mechanism. - **Communication Efficiency**: Reduce communication overhead through sparsification, but it is necessary to ensure that the privacy protection effect will not be sacrificed. ### Solutions The paper proposes the **SparseSecAgg** framework, which solves the above problems by introducing a new gradient sparsification process, called **pairwise sparsification**. The specific methods are as follows: 1. **Pairwise Sparsification**: - Each pair of users shares two random vectors: a pairwise binary multiplication mask (which determines the sparsification pattern) and a pairwise additive mask (which hides the content of the local model). - Each user constructs a sparsified masked model according to the pairwise binary multiplication mask and sends the masked model parameters and their positions to the server. 2. **Mask Cancellation**: - When the sparsified masked models are aggregated on the server side, the pairwise additive masks will cancel each other out, enabling the server to learn the sum of the sparsified local models without revealing the true values of individual models. 3. **Theoretical Analysis**: - The paper conducts a theoretical analysis of SparseSecAgg in terms of convergence, privacy, communication overhead, and computational complexity, quantifying the impact of sparsification on secure aggregation. - By adjusting the sparsification parameter α, a trade - off between privacy protection and communication efficiency can be made. ### Experimental Verification The paper conducts extensive experiments on the CIFAR - 10 and MNIST datasets to verify the effectiveness of SparseSecAgg. The experimental results show that SparseSecAgg significantly reduces communication overhead by up to 7.8 times while maintaining model training performance. ### Summary By introducing the SparseSecAgg framework, the paper successfully solves the communication bottleneck problem in privacy - protected federated learning and achieves the goal of significantly reducing communication overhead without sacrificing privacy protection.

Sparsified Secure Aggregation for Privacy-Preserving Federated Learning

Two-Phase Sparsification with Secure Aggregation for Privacy-Aware Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Efficient and Secure Federated Learning for Financial Applications

An Efficient and Multi-Private Key Secure Aggregation Scheme for Federated Learning

An Efficient and Multi-private Key Secure Aggregation for Federated Learning

Secure Aggregation for Clustered Federated Learning with Passive Adversaries

Fast Secure Aggregation with High Dropout Resilience for Federated Learning

Lightweight Federated Learning Secure Aggregation Protocols

FSSA: Efficient 3-Round Secure Aggregation for Privacy-Preserving Federated Learning

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

Efficient Vertical Federated Learning with Secure Aggregation

Practical and Efficient Secure Aggregation for Privacy-Preserving Machine Learning.

Efficient Secure Aggregation for Privacy-Preserving Federated Machine Learning

Privacy-preserving Decentralized Aggregation for Federated Learning

Split Aggregation: Lightweight Privacy-Preserving Federated Learning Resistant to Byzantine Attacks

Fluent: Round-efficient Secure Aggregation for Private Federated Learning

Boosting Communication Efficiency of Federated Learning's Secure Aggregation

Efficiently Achieving Secure Model Training and Secure Aggregation to Ensure Bidirectional Privacy-Preservation in Federated Learning

Graph Sparsification-Based Secure Federated Learning for Consumer-Driven Internet of Things

An efficient privacy-preserving and verifiable scheme for federated learning