Abstract:Previous state-of-the-art studies have demonstrated that adversaries can access sensitive user data by membership inference attacks (MIAs) in Federated Learning (FL). Introducing differential privacy (DP) into the FL framework is an effective way to enhance the privacy of FL. Nevertheless, in differentially private federated learning (DP-FL), local gradients become excessively sparse in certain training rounds. Especially when training with low privacy budgets, there is a risk of introducing excessive noise into clients’ gradients. This issue can lead to a significant degradation in the accuracy of the global model. Thus, how to balance the user’s privacy and global model accuracy becomes a challenge in DP-FL. To this end, we propose an approach, known as differential privacy federated aggregation, based on significant gradient protection (DP-FedASGP). DP-FedASGP can mitigate excessive noises by protecting significant gradients and accelerate the convergence of the global model by calculating dynamic aggregation weights for gradients. Experimental results show that DP-FedASGP achieves comparable privacy protection effects to DP-FedAvg and cpSGD (communication-private SGD based on gradient quantization) but outperforms DP-FedSNLC (sparse noise based on clipping losses and privacy budget costs) and FedSMP (sparsified model perturbation). Furthermore, the average global test accuracy of DP-FedASGP across four datasets and three models is about 2.62%, 4.71%, 0.45%, and 0.19% higher than the above methods, respectively. These improvements indicate that DP-FedASGP is a promising approach for balancing the privacy and accuracy of DP-FL.

A Stochastic Gradient Descent Algorithm Based on Adaptive Differential Privacy

A(DP)$^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Dynamic Differential-Privacy Preserving SGD

Balancing Privacy and Accuracy Using Significant Gradient Protection in Federated Learning

DPDR: Gradient Decomposition and Reconstruction for Differentially Private Deep Learning

A(DP)$^2$2SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Differentially Private Convolutional Neural Networks with Adaptive Gradient Descent.

An Improved Stochastic Gradient Descent Algorithm Based on Rényi Differential Privacy

An Improved Privacy-Preserving Stochastic Gradient Descent Algorithm

An Adaptive Differential Privacy Method Based on Federated Learning

Enhancing DP-SGD through Non-monotonous Adaptive Scaling Gradient Weight

Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise

A Federated Learning Scheme Meets Dynamic Differential Privacy

Efficient DP-FL: Efficient Differential Privacy Federated Learning Based on Early Stopping Mechanism

Differential Privacy Protection over Deep Learning: an Investigation of Its Impacted Factors

DPAdaMod_AGC: Adaptive Gradient Clipping-Based Differential Privacy

Practical and Private (Deep) Learning without Sampling or Shuffling

A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy

Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent

Differentially Private Federated Learning with an Adaptive Noise Mechanism