Abstract:Federated Learning (FL) has become a cornerstone of privacy protection, shifting the paradigm towards localizing sensitive data while only sending model gradients to a central server. This strategy is designed to reinforce privacy protections and minimize the vulnerabilities inherent in centralized data storage systems. Despite its innovative approach, recent empirical studies have highlighted potential weaknesses in FL, notably regarding the exchange of gradients. In response, this study introduces a novel, efficacious method aimed at safeguarding against gradient leakage, namely, ``AdaDefense". Following the idea that model convergence can be achieved by using different types of optimization methods, we suggest using a local stand-in rather than the actual local gradient for global gradient aggregation on the central server. This proposed approach not only effectively prevents gradient leakage, but also ensures that the overall performance of the model remains largely unaffected. Delving into the theoretical dimensions, we explore how gradients may inadvertently leak private information and present a theoretical framework supporting the efficacy of our proposed method. Extensive empirical tests, supported by popular benchmark experiments, validate that our approach maintains model integrity and is robust against gradient leakage, marking an important step in our pursuit of safe and efficient FL.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the gradient leakage problem in Federated Learning (FL). Although FL enhances privacy protection by sharing only model gradients rather than raw data, recent research shows that these shared gradients can still be used to reconstruct the original training data, thus leaking sensitive information. To solve this problem, the paper proposes a new method named AdaDefense, which aims to prevent gradient leakage while ensuring that the model performance is not significantly affected. ### Specific description of the problem 1. **Risk of gradient leakage**: - In FL, clients send gradients to the central server for global aggregation after training the model locally. - Although this method avoids directly sharing raw data, research shows that by analyzing these gradients, an attacker can partially or completely reconstruct the original training data, thus leaking privacy. 2. **Limitations of existing defense methods**: - Existing defense methods such as gradient perturbation, data obfuscation or encryption techniques can protect privacy to a certain extent, but usually make compromises between privacy protection and computational efficiency. - These methods often require a large amount of computational resources and may reduce the performance of the model. 3. **Innovations of AdaDefense**: - AdaDefense effectively prevents gradient leakage without changing the model architecture or increasing the additional computational burden by introducing a new gradient substitution mechanism. - Specifically, AdaDefense uses the Adam optimization algorithm to process local gradients to generate gradients stand - in, and then uses these substitute gradients for global aggregation. - This method not only protects privacy but also maintains the performance and convergence speed of the model. ### Overview of the solution - **Gradient substitution mechanism**: AdaDefense generates gradient substitutes by processing the original gradients using the Adam optimization algorithm locally. These gradient substitutes retain the high - level feature representation of the gradients but no longer contain the low - level information that can reconstruct the original data. - **Theoretical analysis**: The paper provides a detailed mathematical analysis to prove the effectiveness of the gradient substitution. Specifically, through formula derivation, the author shows how the gradient substitution effectively prevents an attacker from extracting useful information from the gradients. - **Experimental verification**: Through experiments on multiple benchmark datasets, the paper verifies the effectiveness of AdaDefense in preventing gradient leakage while maintaining the performance and accuracy of the model. ### Formula summary - **Adam optimization algorithm**: \[ m_r=\beta_1 m_{r - 1}+(1-\beta_1)g_r \] \[ v_r=\beta_2 v_{r - 1}+(1-\beta_2)g_r^2 \] \[ \hat{m}_r=\frac{m_r}{1-\beta_1^r} \] \[ \hat{v}_r=\frac{v_r}{1-\beta_2^r} \] \[ \hat{g}_r=\frac{\hat{m}_r}{\sqrt{\hat{v}_r}+\epsilon} \] - **Derivative of gradient substitution**: \[ \frac{\partial\hat{g}_r}{\partial g_r}=\frac{(1-\beta_1)(V +\epsilon)-((1-\beta_2)\beta_1 m_{r - 1}+(1-\beta_2)(1-\beta_1)g_r)}{(1-\beta_1^r)(V+\epsilon)^2} \] where \(V = \sqrt{\hat{v}_r}\) Through these formulas, the paper explains in detail how AdaDefense effectively prevents gradient leakage without affecting the model performance, thereby enhancing the privacy security of the FL system.

Gradients Stand-in for Defending Deep Leakage in Federated Learning

Gradient Leakage Defense with Key-Lock Module for Federated Learning

Quantification of the Leakage in Federated Learning

A Framework for Evaluating Gradient Leakage Attacks in Federated Learning

Optimal Defenses Against Gradient Reconstruction Attacks

Shield Against Gradient Leakage Attacks: Adaptive Privacy-Preserving Federated Learning

Secure Federated Learning with Model Compression.

Gradient-Leakage Resilient Federated Learning

Concealing Sensitive Samples against Gradient Leakage in Federated Learning

Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning

FedDAA: a robust federated learning framework to protect privacy and defend against adversarial attack

FedDef: Defense Against Gradient Leakage in Federated Learning-Based Network Intrusion Detection Systems

A New Federated Learning Framework Against Gradient Inversion Attacks

Gradient-based Defense Methods for Data Leakage in Vertical Federated Learning

Defense Against Gradient Leakage Attacks via Learning to Obscure Data

FL-AGN: A Privacy-Enhanced Federated Learning Method Based on Adaptive Gaussian Noise for Resisting Gradient Inference Attacks

Gradient Leakage Attacks in Federated Learning: Research Frontiers, Taxonomy, and Future Directions

Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective

Random Gradient Masking as a Defensive Measure to Deep Leakage in Federated Learning

Gradient leakage attacks in federated learning

Does Differential Privacy Really Protect Federated Learning from Gradient Leakage Attacks?