Abstract:In personalized federated learning (PFL), it is widely recognized that achieving both high model generalization and effective personalization poses a significant challenge due to their conflicting nature. As a result, existing PFL methods can only manage a trade-off between these two objectives. This raises an interesting question: Is it feasible to develop a model capable of achieving both objectives simultaneously? Our paper presents an affirmative answer, and the key lies in the observation that deep models inherently exhibit hierarchical architectures, which produce representations with various levels of generalization and personalization at different stages. A straightforward approach stemming from this observation is to select multiple representations from these layers and combine them to concurrently achieve generalization and personalization. However, the number of candidate representations is commonly huge, which makes this method infeasible due to high computational <a class="link-external link-http" href="http://costs.To" rel="external noopener nofollow">this http URL</a> address this problem, we propose DualFed, a new method that can directly yield dual representations correspond to generalization and personalization respectively, thereby simplifying the optimization task. Specifically, DualFed inserts a personalized projection network between the encoder and classifier. The pre-projection representations are able to capture generalized information shareable across clients, and the post-projection representations are effective to capture task-specific information on local clients. This design minimizes the mutual interference between generalization and personalization, thereby achieving a win-win situation. Extensive experiments show that DualFed can outperform other FL methods. Code is available at <a class="link-external link-https" href="https://github.com/GuogangZhu/DualFed" rel="external noopener nofollow">this https URL</a>.

Feed: Towards Personalization-Effective Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

FedL2P: Federated Learning to Personalize

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing

Group Personalized Federated Learning

Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training

Pfedgf: Enabling Personalized Federated Learning Via Gradient Fusion

Personalized federated learning based on feature fusion

CFedPer: Clustered Federated Learning with Two-Stages Optimization for Personalization

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations

Efficient Hierarchical Personalized Federated Learning on Heterogeneous Data

Personalized Federated Learning with Domain Generalization

Federated Learning with Personalization Layers

Sparse Personalized Federated Learning

Efficient Personalized Federated Learning on Selective Model Training

Exploring Personalization Via Federated Representation Learning on Non-Iid Data.

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning