Abstract:In personalized federated learning (PFL), it is widely recognized that achieving both high model generalization and effective personalization poses a significant challenge due to their conflicting nature. As a result, existing PFL methods can only manage a trade-off between these two objectives. This raises an interesting question: Is it feasible to develop a model capable of achieving both objectives simultaneously? Our paper presents an affirmative answer, and the key lies in the observation that deep models inherently exhibit hierarchical architectures, which produce representations with various levels of generalization and personalization at different stages. A straightforward approach stemming from this observation is to select multiple representations from these layers and combine them to concurrently achieve generalization and personalization. However, the number of candidate representations is commonly huge, which makes this method infeasible due to high computational <a class="link-external link-http" href="http://costs.To" rel="external noopener nofollow">this http URL</a> address this problem, we propose DualFed, a new method that can directly yield dual representations correspond to generalization and personalization respectively, thereby simplifying the optimization task. Specifically, DualFed inserts a personalized projection network between the encoder and classifier. The pre-projection representations are able to capture generalized information shareable across clients, and the post-projection representations are effective to capture task-specific information on local clients. This design minimizes the mutual interference between generalization and personalization, thereby achieving a win-win situation. Extensive experiments show that DualFed can outperform other FL methods. Code is available at <a class="link-external link-https" href="https://github.com/GuogangZhu/DualFed" rel="external noopener nofollow">this https URL</a>.

Pfedgf: Enabling Personalized Federated Learning Via Gradient Fusion

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Personalized federated learning based on feature fusion

PGFed: Personalize Each Client's Global Objective for Federated Learning

Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients

Gradient Free Personalized Federated Learning

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Personalized Federated Learning under Mixture of Distributions

Personalized Federated Learning with Multi-View Geometry Structure

Feed: Towards Personalization-Effective Federated Learning

Decentralized Directed Collaboration for Personalized Federated Learning

GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Personalized Federated Learning on Long-Tailed Data Via Adversarial Feature Augmentation.

Efficient Personalized Federated Learning via Sparse Model-Adaptation

Personalized Federated Learning with Domain Generalization

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations

pFedKT: Personalized federated learning with dual knowledge transfer