Abstract:In personalized federated learning (PFL), it is widely recognized that achieving both high model generalization and effective personalization poses a significant challenge due to their conflicting nature. As a result, existing PFL methods can only manage a trade-off between these two objectives. This raises an interesting question: Is it feasible to develop a model capable of achieving both objectives simultaneously? Our paper presents an affirmative answer, and the key lies in the observation that deep models inherently exhibit hierarchical architectures, which produce representations with various levels of generalization and personalization at different stages. A straightforward approach stemming from this observation is to select multiple representations from these layers and combine them to concurrently achieve generalization and personalization. However, the number of candidate representations is commonly huge, which makes this method infeasible due to high computational <a class="link-external link-http" href="http://costs.To" rel="external noopener nofollow">this http URL</a> address this problem, we propose DualFed, a new method that can directly yield dual representations correspond to generalization and personalization respectively, thereby simplifying the optimization task. Specifically, DualFed inserts a personalized projection network between the encoder and classifier. The pre-projection representations are able to capture generalized information shareable across clients, and the post-projection representations are effective to capture task-specific information on local clients. This design minimizes the mutual interference between generalization and personalization, thereby achieving a win-win situation. Extensive experiments show that DualFed can outperform other FL methods. Code is available at <a class="link-external link-https" href="https://github.com/GuogangZhu/DualFed" rel="external noopener nofollow">this https URL</a>.

FedFTHA: A Fine-Tuning and Head Aggregation Method in Federated Learning

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedAH: Aggregated Head for Personalized Federated Learning

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

FedALA: Adaptive Local Aggregation for Personalized Federated Learning

Personalized Federated Learning with Local Optimization Models

Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning

Improving Global Generalization and Local Personalization for Federated Learning

UPFL: Unsupervised Personalized Federated Learning Towards New Clients

Pfedbea: Combatting Data Heterogeneity for Personalized Federated Learning by Body Exchange and Aggregation Abandon

FedAS: Bridging Inconsistency in Personalized Federated Learning

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

Personalized federated learning based on multi-head attention algorithm

Personalized federated learning based on feature fusion

Sparse Federated Learning with Hierarchical Personalization Models.

To Be Global or Personalized: Generalized Federated Learning with Cooperative Adaptation for Data Heterogeneity

Personalized Federated Learning with First Order Model Optimization

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations

FedPFT: Federated Proxy Fine-Tuning of Foundation Models

FedGH: Heterogeneous Federated Learning with Generalized Global Header