Abstract:Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning methods suffer from inflexible personalization and convergence difficulties due to two main factors: 1) Firstly, we observe that the prevailing personalization methods mainly achieve this by personalizing a fixed portion of the model, which lacks flexibility. 2) Moreover, we further demonstrate that the default gradient calculation is sensitive to the widely-used clipping operations in differential privacy, resulting in difficulties in convergence. Considering that Fisher information values can serve as an effective measure for estimating the information content of parameters by reflecting the model sensitivity to parameters, we aim to leverage this property to address the aforementioned challenges. In this paper, we propose a novel federated learning method with D ynamic Fisher P ersonalization and A daptive Constraint (FedDPA) to handle these challenges. Firstly, by using layer-wise Fisher information to measure the information content of local parameters, we retain local parameters with high Fisher values during the personalization process, which are considered informative, simultaneously prevent these parameters from noise perturbation. Secondly, we introduce an adaptive approach by applying differential constraint strategies to personalized parameters and shared parameters identified in the previous for better convergence. Our method boosts performance through flexible personalization while mitigating the slow convergence caused by clipping operations. Experimental results on CIFAR-10, FEMNIST and SVHN dataset demonstrate the effectiveness of our approach in achieving better performance and robustness against clipping, under personalized federated learning with differential privacy.

Federated Learning with Personalized Differential Privacy Combining Client Selection

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Personalized Federated Learning With Differential Privacy and Convergence Guarantee

PLDP-FL: Federated Learning with Personalized Local Differential Privacy

The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy

Enhancing Federated Learning with Adaptive Differential Privacy and Priority-Based Aggregation

Differentially Private Federated Learning with an Adaptive Noise Mechanism

A differential privacy federated learning framework for accelerating convergence

Optimizing the Numbers of Queries and Replies in Convex Federated Learning with Differential Privacy

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

Dynamic Personalized Federated Learning with Adaptive Differential Privacy

FDP-FL: Differentially Private Federated Learning with Flexible Privacy Budget Allocation

Joint Communication and Learning Design of Differential Privacy for Federated Learning over Multi-cell Networks.

Efficient DP-FL: Efficient Differential Privacy Federated Learning Based on Early Stopping Mechanism

Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence

On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times

FedDP-SA: Boosting Differentially Private Federated Learning Via Local Dataset Splitting

QI-DPFL: Quality-Aware and Incentive-Boosted Federated Learning with Differential Privacy

Decentralized Directed Collaboration for Personalized Federated Learning

LDP-Fed+: A robust and privacy-preserving federated learning based classification framework enabled by local differential privacy

Adaptive Differential Privacy in Federated Learning: A Priority-Based Approach