Abstract:The generation of large amounts of personal data provides data centers with sufficient resources to mine idiosyncrasy from private records. User modeling has long been a fundamental task with the goal of capturing the latent characteristics of users from their behaviors. However, centralized user modeling on collected data has raised concerns about the risk of data misuse and privacy leakage. As a result, federated user modeling has come into favor, since it expects to provide secure multi-client collaboration for user modeling through federated learning. Unfortunately, to the best of our knowledge, existing federated learning methods that ignore the inconsistency among clients cannot be applied directly to practical user modeling scenarios, and moreover, they meet the following critical challenges: 1) Statistical heterogeneity . The distributions of user data in different clients are not always independently identically distributed (IID), which leads to unique clients with needful personalized information; 2) Privacy heterogeneity . User data contains both public and private information, which have different levels of privacy, indicating that we should balance different information shared and protected; 3) Model heterogeneity . The local user models trained with client records are heterogeneous, and thus require a flexible aggregation in the server; 4) Quality heterogeneity . Low-quality information from inconsistent clients poisons the reliability of user models and offsets the benefit from high-quality ones, meaning that we should augment the high-quality information during the process. To address the challenges, in this paper, we first propose a novel client-server architecture framework, namely Hierarchical Personalized Federated Learning (HPFL), with a primary goal of serving federated learning for user modeling in inconsistent clients. More specifically, the client train and deliver the local user model via the hierarchical components containing hierarchical information from privacy heterogeneity to join collaboration in federated learning. Moreover, the client updates the personalized user model with a fine-grained personalized update strategy for statistical heterogeneity. Correspondingly, the server flexibly aggregates hierarchical components from heterogeneous user models in the case of privacy and model heterogeneity with a differentiated component aggregation strategy. In order to augment high-quality information and generate high-quality user models, we expand HPFL to the Augmented-HPFL (AHPFL) framework by incorporating the augmented mechanisms, which filters out low-quality information such as noise, sparse information and redundant information. Specially, we construct two implementations of AHPFL, i.e., AHPFL-SVD and AHPFL-AE, where the augmented mechanisms follow SVD (singular value decomposition) and AE (autoencoder), respectively. Finally, we conduct extensive experiments on real-world datasets, which demonstrate the effectiveness of both HPFL and AHPFL frameworks.

FedPHP: Federated Personalization with Inherited Private Models

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

Sparse Federated Learning with Hierarchical Personalized Models

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

FedL2P: Federated Learning to Personalize

A Personalized Federated Learning Method Based on Clustering and Knowledge Distillation

Hierarchical Personalized Federated Learning for User Modeling

Group Personalized Federated Learning

Federated User Modeling from Hierarchical Information

To Be Global or Personalized: Generalized Federated Learning with Cooperative Adaptation for Data Heterogeneity

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

pFedKT: Personalized federated learning with dual knowledge transfer

The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation

Personalized federated learning based on feature fusion

Learn What You Need in Personalized Federated Learning

pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning

pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learning

Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneity

PPFL: A Personalized Federated Learning Framework for Heterogeneous Population

Towards Personalized Federated Learning via Comprehensive Knowledge Distillation

Tackling Data Heterogeneity in Federated Learning with Class Prototypes