Abstract:Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning methods suffer from inflexible personalization and convergence difficulties due to two main factors: 1) Firstly, we observe that the prevailing personalization methods mainly achieve this by personalizing a fixed portion of the model, which lacks flexibility. 2) Moreover, we further demonstrate that the default gradient calculation is sensitive to the widely-used clipping operations in differential privacy, resulting in difficulties in convergence. Considering that Fisher information values can serve as an effective measure for estimating the information content of parameters by reflecting the model sensitivity to parameters, we aim to leverage this property to address the aforementioned challenges. In this paper, we propose a novel federated learning method with D ynamic Fisher P ersonalization and A daptive Constraint (FedDPA) to handle these challenges. Firstly, by using layer-wise Fisher information to measure the information content of local parameters, we retain local parameters with high Fisher values during the personalization process, which are considered informative, simultaneously prevent these parameters from noise perturbation. Secondly, we introduce an adaptive approach by applying differential constraint strategies to personalized parameters and shared parameters identified in the previous for better convergence. Our method boosts performance through flexible personalization while mitigating the slow convergence caused by clipping operations. Experimental results on CIFAR-10, FEMNIST and SVHN dataset demonstrate the effectiveness of our approach in achieving better performance and robustness against clipping, under personalized federated learning with differential privacy.

A Federated Learning Scheme Meets Dynamic Differential Privacy

Client-based differential privacy federated learning

An Adaptive Differential Privacy Method Based on Federated Learning

A Differential Privacy Strategy Based on Local Features of Non-Gaussian Noise in Federated Learning

A Novel Federated Learning with Bidirectional Adaptive Differential Privacy.

Federated Transfer Learning with Differential Privacy

Towards Efficient and Privacy-Preserving Federated Deep Learning

Federated learning with differential privacy via fast Fourier transform for tighter-efficient combining

Dynamic Personalized Federated Learning with Adaptive Differential Privacy

Efficient federated learning privacy preservation method with heterogeneous differential privacy

Utility Optimization of Federated Learning with Differential Privacy

The Impact of Differential Privacy on Model Fairness in Federated Learning.

A Privacy-Enhanced Federated Learning Model Training Method

Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy

Model Level Contrastive Federated Learning with Differential Privacy

A Robust Game-theoretical Federated Learning Framework with Joint Differential Privacy

Local Differential Privacy-Based Federated Learning under Personalized Settings

An Efficient Federated Convolutional Neural Network Scheme with Differential Privacy.

FedFDP: Fairness-Aware Federated Learning with Differential Privacy

A Systematic Survey for Differential Privacy Techniques in Federated Learning