Abstract:In recent years, personalized federated learning (pFL) has attracted increasing attention for its potential in dealing with statistical heterogeneity among clients. However, the state-of-the-art pFL methods rely on model parameters aggregation at the server side, which require all models to have the same structure and size, and thus limits the application for more heterogeneous scenarios. To deal with such model constraints, we exploit the potentials of heterogeneous model settings and propose a novel training framework to employ personalized models for different clients. Specifically, we formulate the aggregation procedure in original pFL into a personalized group knowledge transfer training algorithm, namely, KT-pFL, which enables each client to maintain a personalized soft prediction at the server side to guide the others' local training. KT-pFL updates the personalized soft prediction of each client by a linear combination of all local soft predictions using a knowledge coefficient matrix, which can adaptively reinforce the collaboration among clients who own similar data distribution. Furthermore, to quantify the contributions of each client to others' personalized training, the knowledge coefficient matrix is parameterized so that it can be trained simultaneously with the models. The knowledge coefficient matrix and the model parameters are alternatively updated in each round following the gradient descent way. Extensive experiments on various datasets (EMNIST, Fashion\_MNIST, CIFAR-10) are conducted under different settings (heterogeneous models and data distributions). It is demonstrated that the proposed framework is the first federated learning paradigm that realizes personalized model training via parameterized group knowledge transfer while achieving significant performance gain comparing with state-of-the-art algorithms.

Multiple Adaptive Prototypes Learning in Personalized Federated Learning

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Tackling Data Heterogeneity in Federated Learning with Class Prototypes

Parameterized Knowledge Transfer for Personalized Federated Learning

Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneity

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Rethinking Personalized Federated Learning from Knowledge Perspective

Feed: Towards Personalization-Effective Federated Learning

Dual Calibration-based Personalised Federated Learning

pFedKT: Personalized federated learning with dual knowledge transfer

Cooperative Multi-Model Training for Personalized Federated Learning over Heterogeneous Devices

FedMAP: Unlocking Potential in Personalized Federated Learning through Bi-Level MAP Optimization

Pfedlhns: Personalized Federated Learning Via Local Hypernetworks

Adaptive Personalized Federated Learning for Heterogeneous Data: a Method Based on Parameter Decomposition and Continual Learning

A Unified Personalized Federated Learning Framework Ensuring Domain Generalization

Group Personalized Federated Learning

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

FedL2P: Federated Learning to Personalize

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing