Abstract:Federated learning (FL) has attracted increasing attention in recent years due to its data privacy preservation and great applicability to large-scale user scenarios. However, when FL faces numerous clients, it is inevitable to emerge the non-independent and identically distributed (non-iid) data between clients, which brings an enormous challenge for model training and performance analysis like convergence. Besides, due to the non-iid data, the participating clients of FL tend to be extremely heterogeneous so the number of samplings among clients causes a sampling variance problem, which induces a huge variation in convergence. More importantly, although FL can foster privacy security via locally retaining the training data, if local data is secret and sensitive, FL should have more powerful privacy protection to resist the cloud server or third party to infer private information from shared models or intermediate gradients. Facing the non-iid and privacy challenges, we propose a differential privacy (DP) based non-iid FL algorithm called DPNFL to jointly tackle these two issues. Specifically, motivated by the DP and its variants, we are the first to adopt the truncated concentrated differential privacy technique under the FL scenario to more tightly track end-to-end privacy loss, while requiring less noise injection for the same level of DP. To avoid the sampling variance problem, we enable the server to sample the partial clients uniformly without replacement, which also guarantees unbiased sampling. To further improve the algorithm performance, we also propose an adaptive version of DPNFL named AdDPNFL, which adopts the adaptive optimization on the server-side to simultaneously alleviate the impact of non-iid data and DP noise on model utility. Finally, we perform extensive experiments to validate the effectiveness and superiority of our algorithms.

Differentially Private Federated Learning with an Adaptive Noise Mechanism

Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

A Fine-Grained Differentially Private Federated Learning Against Leakage from Gradients

Adaptive Differential Privacy in Federated Learning: A Priority-Based Approach

Differentially Private Federated Learning on Non-iid Data: Convergence Analysis and Adaptive Optimization

Differentially Private Federated Learning: Algorithm, Analysis and Optimization

Idp-Fl: A Fine-Grained and Privacy-Aware Federated Learning Framework for Deep Neural Networks

FDP-FL: Differentially Private Federated Learning with Flexible Privacy Budget Allocation

Differentially Private Federated Learning with Local Momentum Updates and Gradients Filtering

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

FedDP-SA: Boosting Differentially Private Federated Learning Via Local Dataset Splitting

ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations

Efficient DP-FL: Efficient Differential Privacy Federated Learning Based on Early Stopping Mechanism

Communication-Efficient and Privacy-Preserving Aggregation in Federated Learning With Adaptability

Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning

Differentially Private Byzantine-Robust Federated Learning

Aldp-fl: an adaptive local differential privacy-based federated learning mechanism for IoT

A Federated Learning Framework Based on Differentially Private Continuous Data Release

Federated Learning with Personalized Differential Privacy Combining Client Selection

Mitigating Noise Detriment in Differentially Private Federated Learning with Model Pre-training