Abstract:Federated learning, as a distributed machine learning framework, enables clients to conduct model training without transmitting their data to the server, which is used to solve the dilemma of data silos and data privacy. It can work well on clients having similar data characteristics and distribution. However, it has some limitations where the dataset of clients may be different in distribution, quantity, and concept in many application scenarios. Personalized federated learning (PFL) is a new federated learning paradigm that aims to guarantee client personalized models’ effectiveness when collaborating with the cloud server. Intuitively, providing further facilitated collaborations for the clients with similar data characteristics and distribution can benefit personalized model building. However, due to the invisibility of client data, it is challenging to extract client characteristics and define collaborative relationships among them from a fine-grained view. Moreover, a reasonable collaborative training approach needs to be designed for a distributed server-client framework. In this paper, we design a H ierarchical A ttention-enhanced M eta-learning Network ( HAM for short) to address this issue. The main advantage of HAM is that it utilizes the meta-learning approach of taking model parameters as features and learns to learn an extra model for each client to analyze similarities according to their local dataset automatically. According to its two-layers framework, HAM can reasonably achieve a trade-off between clients’ personality and commonality, and provides a hybrid model with useful information from all clients. Considering there are two networks ( HAM and base network) that need to learn for each client during the federated training process, we then provide an alternative learning approach to train them in an end-to-end fashion. To further clarify the approach, we describe the personalized federated learning settings framework as FedHAM where HAM network is distributed deployed in each client. Extensive experiments based on two datasets prove that our method outperforms state-of-the-art baselines under different evaluation metrics.

Personalized Federated Learning via Amortized Bayesian Meta-Learning

Personalized Federated Learning: A Meta-Learning Approach

Personalized Federated Learning with Layer-Wise Feature Transformation Via Meta-Learning

Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering

Personalized Federated Learning with Hidden Information on Personalized Prior

Federated mutual learning

Personalized Federated Learning: A Clustered Distributed Co-Meta-Learning Approach

Learn What You Need in Personalized Federated Learning

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Personalized federated learning based on feature fusion

FedMetaMed: Federated Meta-Learning for Personalized Medication in Distributed Healthcare Systems

Personalized Federated Learning through Local Memorization

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

FED-3DA: A Dynamic and Personalized Federated Learning Framework

Configure Your Federation: Hierarchical Attention-enhanced Meta-Learning Network for Personalized Federated Learning

Personalized Federated Learning with Moreau Envelopes

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

Few-Shot Model Agnostic Federated Learning

Confidence-aware Personalized Federated Learning via Variational Expectation Maximization

FedFTHA: A Fine-Tuning and Head Aggregation Method in Federated Learning

Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach