Abstract:Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning methods suffer from inflexible personalization and convergence difficulties due to two main factors: 1) Firstly, we observe that the prevailing personalization methods mainly achieve this by personalizing a fixed portion of the model, which lacks flexibility. 2) Moreover, we further demonstrate that the default gradient calculation is sensitive to the widely-used clipping operations in differential privacy, resulting in difficulties in convergence. Considering that Fisher information values can serve as an effective measure for estimating the information content of parameters by reflecting the model sensitivity to parameters, we aim to leverage this property to address the aforementioned challenges. In this paper, we propose a novel federated learning method with D ynamic Fisher P ersonalization and A daptive Constraint (FedDPA) to handle these challenges. Firstly, by using layer-wise Fisher information to measure the information content of local parameters, we retain local parameters with high Fisher values during the personalization process, which are considered informative, simultaneously prevent these parameters from noise perturbation. Secondly, we introduce an adaptive approach by applying differential constraint strategies to personalized parameters and shared parameters identified in the previous for better convergence. Our method boosts performance through flexible personalization while mitigating the slow convergence caused by clipping operations. Experimental results on CIFAR-10, FEMNIST and SVHN dataset demonstrate the effectiveness of our approach in achieving better performance and robustness against clipping, under personalized federated learning with differential privacy.

Federated Learning with Personalization Layers

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

FedEmbed: Personalized Private Federated Learning

Personalized Multi-tier Federated Learning

Personalized federated learning based on feature fusion

FedL2P: Federated Learning to Personalize

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework

Personalized Federated Learning through Local Memorization

Ensuring Fairness and Gradient Privacy in Personalized Heterogeneous Federated Learning

Sparse Personalized Federated Learning

Three Approaches for Personalization with Applications to Federated Learning

Personalized Federated Learning via Stacking

Group Personalized Federated Learning

Learn What You Need in Personalized Federated Learning

PerFED-GAN: Personalized Federated Learning via Generative Adversarial Networks

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Dynamic Personalized Federated Learning with Adaptive Differential Privacy

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization

Personalized Federated Learning Techniques: Empirical Analysis

Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients