Abstract:In personalized federated learning (PFL), it is widely recognized that achieving both high model generalization and effective personalization poses a significant challenge due to their conflicting nature. As a result, existing PFL methods can only manage a trade-off between these two objectives. This raises an interesting question: Is it feasible to develop a model capable of achieving both objectives simultaneously? Our paper presents an affirmative answer, and the key lies in the observation that deep models inherently exhibit hierarchical architectures, which produce representations with various levels of generalization and personalization at different stages. A straightforward approach stemming from this observation is to select multiple representations from these layers and combine them to concurrently achieve generalization and personalization. However, the number of candidate representations is commonly huge, which makes this method infeasible due to high computational <a class="link-external link-http" href="http://costs.To" rel="external noopener nofollow">this http URL</a> address this problem, we propose DualFed, a new method that can directly yield dual representations correspond to generalization and personalization respectively, thereby simplifying the optimization task. Specifically, DualFed inserts a personalized projection network between the encoder and classifier. The pre-projection representations are able to capture generalized information shareable across clients, and the post-projection representations are effective to capture task-specific information on local clients. This design minimizes the mutual interference between generalization and personalization, thereby achieving a win-win situation. Extensive experiments show that DualFed can outperform other FL methods. Code is available at <a class="link-external link-https" href="https://github.com/GuogangZhu/DualFed" rel="external noopener nofollow">this https URL</a>.

Hypernetwork-Driven Model Fusion for Federated Domain Generalization

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalization

HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learning

Learning to Generalize in Heterogeneous Federated Networks

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations

FedGH: Heterogeneous Federated Learning with Generalized Global Header

Federated Domain Generalization with Generalization Adjustment

HyperFedNet: Communication-Efficient Personalized Federated Learning Via Hypernetwork

Reducing Spurious Correlation for Federated Domain Generalization

Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization

HyperFeel: an Efficient Federated Learning Framework Using Hyperdimensional Computing

FedWon: Triumphing Multi-domain Federated Learning Without Normalization

Improving Group Connectivity for Generalization of Federated Deep Learning

FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent

Handling Data Heterogeneity in Federated Learning via Knowledge Distillation and Fusion

Contrastive-enhanced Domain Generalization with Federated Learning

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Federated Learning with Data-Agnostic Distribution Fusion

Bilateral Improvement in Local Personalization and Global Generalization in Federated Learning.