Abstract:The generation of large amounts of personal data provides data centers with sufficient resources to mine idiosyncrasy from private records. User modeling has long been a fundamental task with the goal of capturing the latent characteristics of users from their behaviors. However, centralized user modeling on collected data has raised concerns about the risk of data misuse and privacy leakage. As a result, federated user modeling has come into favor, since it expects to provide secure multi-client collaboration for user modeling through federated learning. Unfortunately, to the best of our knowledge, existing federated learning methods that ignore the inconsistency among clients cannot be applied directly to practical user modeling scenarios, and moreover, they meet the following critical challenges: 1) Statistical heterogeneity . The distributions of user data in different clients are not always independently identically distributed (IID), which leads to unique clients with needful personalized information; 2) Privacy heterogeneity . User data contains both public and private information, which have different levels of privacy, indicating that we should balance different information shared and protected; 3) Model heterogeneity . The local user models trained with client records are heterogeneous, and thus require a flexible aggregation in the server; 4) Quality heterogeneity . Low-quality information from inconsistent clients poisons the reliability of user models and offsets the benefit from high-quality ones, meaning that we should augment the high-quality information during the process. To address the challenges, in this paper, we first propose a novel client-server architecture framework, namely Hierarchical Personalized Federated Learning (HPFL), with a primary goal of serving federated learning for user modeling in inconsistent clients. More specifically, the client train and deliver the local user model via the hierarchical components containing hierarchical information from privacy heterogeneity to join collaboration in federated learning. Moreover, the client updates the personalized user model with a fine-grained personalized update strategy for statistical heterogeneity. Correspondingly, the server flexibly aggregates hierarchical components from heterogeneous user models in the case of privacy and model heterogeneity with a differentiated component aggregation strategy. In order to augment high-quality information and generate high-quality user models, we expand HPFL to the Augmented-HPFL (AHPFL) framework by incorporating the augmented mechanisms, which filters out low-quality information such as noise, sparse information and redundant information. Specially, we construct two implementations of AHPFL, i.e., AHPFL-SVD and AHPFL-AE, where the augmented mechanisms follow SVD (singular value decomposition) and AE (autoencoder), respectively. Finally, we conduct extensive experiments on real-world datasets, which demonstrate the effectiveness of both HPFL and AHPFL frameworks.

Personalized Federated Learning via Sequential Layer Expansion in Representation Learning

Exploring Personalization Via Federated Representation Learning on Non-Iid Data.

Learn What You Need in Personalized Federated Learning

Federated Learning with Personalization Layers

Towards Optimal Customized Architecture for Heterogeneous Federated Learning with Contrastive Cloud-Edge Model Decoupling

Personalized federated learning based on feature fusion

Personalized Federated Learning with Layer-Wise Feature Transformation Via Meta-Learning

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

Exploiting Shared Representations for Personalized Federated Learning

Personalized Federated Reinforcement Learning: Balancing Personalization and Experience Sharing Via Distance Constraint

Personalized Federated Learning with Feature Alignment and Classifier Collaboration

Decentralized Personalized Online Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Federated User Modeling from Hierarchical Information

Personalized Federated Learning via Amortized Bayesian Meta-Learning

Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneity

Personalized Federated Learning with First Order Model Optimization

Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data

A Closer Look at Personalization in Federated Image Classification

A Unified Personalized Federated Learning Framework Ensuring Domain Generalization