Abstract:Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning methods suffer from inflexible personalization and convergence difficulties due to two main factors: 1) Firstly, we observe that the prevailing personalization methods mainly achieve this by personalizing a fixed portion of the model, which lacks flexibility. 2) Moreover, we further demonstrate that the default gradient calculation is sensitive to the widely-used clipping operations in differential privacy, resulting in difficulties in convergence. Considering that Fisher information values can serve as an effective measure for estimating the information content of parameters by reflecting the model sensitivity to parameters, we aim to leverage this property to address the aforementioned challenges. In this paper, we propose a novel federated learning method with D ynamic Fisher P ersonalization and A daptive Constraint (FedDPA) to handle these challenges. Firstly, by using layer-wise Fisher information to measure the information content of local parameters, we retain local parameters with high Fisher values during the personalization process, which are considered informative, simultaneously prevent these parameters from noise perturbation. Secondly, we introduce an adaptive approach by applying differential constraint strategies to personalized parameters and shared parameters identified in the previous for better convergence. Our method boosts performance through flexible personalization while mitigating the slow convergence caused by clipping operations. Experimental results on CIFAR-10, FEMNIST and SVHN dataset demonstrate the effectiveness of our approach in achieving better performance and robustness against clipping, under personalized federated learning with differential privacy.

PerFreezeClip: Personalized Federated Learning Based on Adaptive Clipping

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

FL-Clip: Bridging Plasticity and Stability in Pre-Trained Federated Class-Incremental Learning Models

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Federated Learning with Personalization Layers

Gradient Free Personalized Federated Learning

Dynamic Personalized Federated Learning with Adaptive Differential Privacy

Efficient Hierarchical Personalized Federated Learning on Heterogeneous Data

Pfedgf: Enabling Personalized Federated Learning Via Gradient Fusion

A novel parameter decoupling approach of personalised federated learning for image analysis

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Feed: Towards Personalization-Effective Federated Learning

FedPE: Adaptive Model Pruning-Expanding for Federated Learning on Mobile Devices

CFedPer: Clustered Federated Learning with Two-Stages Optimization for Personalization

FedFDP: Fairness-Aware Federated Learning with Differential Privacy

Personalized Federated Learning with Hidden Information on Personalized Prior

Preconditioned Federated Learning

Personalized Federated Learning With Differential Privacy and Convergence Guarantee

FedQClip: Accelerating Federated Learning Via Quantized Clipped SGD