Abstract:Federated learning, as a distributed machine learning framework, enables clients to conduct model training without transmitting their data to the server, which is used to solve the dilemma of data silos and data privacy. It can work well on clients having similar data characteristics and distribution. However, it has some limitations where the dataset of clients may be different in distribution, quantity, and concept in many application scenarios. Personalized federated learning (PFL) is a new federated learning paradigm that aims to guarantee client personalized models’ effectiveness when collaborating with the cloud server. Intuitively, providing further facilitated collaborations for the clients with similar data characteristics and distribution can benefit personalized model building. However, due to the invisibility of client data, it is challenging to extract client characteristics and define collaborative relationships among them from a fine-grained view. Moreover, a reasonable collaborative training approach needs to be designed for a distributed server-client framework. In this paper, we design a H ierarchical A ttention-enhanced M eta-learning Network ( HAM for short) to address this issue. The main advantage of HAM is that it utilizes the meta-learning approach of taking model parameters as features and learns to learn an extra model for each client to analyze similarities according to their local dataset automatically. According to its two-layers framework, HAM can reasonably achieve a trade-off between clients’ personality and commonality, and provides a hybrid model with useful information from all clients. Considering there are two networks ( HAM and base network) that need to learn for each client during the federated training process, we then provide an alternative learning approach to train them in an end-to-end fashion. To further clarify the approach, we describe the personalized federated learning settings framework as FedHAM where HAM network is distributed deployed in each client. Extensive experiments based on two datasets prove that our method outperforms state-of-the-art baselines under different evaluation metrics.

A personalized federated learning method based on the residual multi-head attention mechanism

FedDGP: Disentangling Global and Personal Models for Federated Learning

Personalized Federated Learning with Local Attention

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

Personalized federated learning based on multi-head attention algorithm

FedAH: Aggregated Head for Personalized Federated Learning

Personalized Federated Learning on Long-Tailed Data Via Adversarial Feature Augmentation.

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Dual Calibration-based Personalised Federated Learning

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Group Personalized Federated Learning

Personalized federated learning based on feature fusion

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Personalized Federated Learning Techniques: Empirical Analysis

Configure Your Federation: Hierarchical Attention-enhanced Meta-Learning Network for Personalized Federated Learning

Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning

PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization

Personalized Federated Learning via Feature Distribution Adaptation

FedL2P: Federated Learning to Personalize

FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data