Abstract:Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at <a class="link-external link-https" href="https://github.com/ljaiverson/pgfed" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the poor performance of traditional Federated Learning (FL) when dealing with heterogeneous data. Specifically, traditional federated learning methods (such as FedAvg) coordinate the data of participating clients by training a single global consensus model. However, this approach leads to issues with convergence and poor generalization performance when faced with non-independent and identically distributed (Non-IID) data. To solve this problem, the paper proposes a new personalized federated learning framework—Personalized Global Federated Learning (PGFed), which allows each client to explicitly personalize its global objective based on its own and other clients' risks. ### Key Points Summary 1. **Problem Background**: - Traditional federated learning performs poorly when handling heterogeneous data. - Non-independent and identically distributed (Non-IID) data lead to decreased model convergence and generalization performance. 2. **Solution**: - Propose the PGFed framework, allowing each client to explicitly personalize its global objective. - Improve model adaptability and generalization by explicitly aggregating the empirical risks among clients. 3. **Technical Contributions**: - **Explicit Personalization**: Improve model adaptability and generalization by explicitly aggregating the empirical risks among clients. - **Communication Efficiency**: Avoid high communication costs by using first-order approximation to estimate non-local risks. - **Momentum Upgrade**: Introduce a momentum mechanism (PGFedMo) to further enhance model training efficiency. 4. **Experimental Results**: - PGFed and PGFedMo significantly outperform existing personalized federated learning methods on multiple datasets and different federated settings, with accuracy improvements of up to 15.47%. ### Core Innovations of the Paper - **Explicit Personalization**: Unlike existing methods that implicitly transfer collaborative knowledge, PGFed explicitly aggregates the empirical risks among clients, allowing each client to more directly utilize information from other clients. - **Communication Efficiency**: By using first-order approximation to estimate non-local risks, it avoids \(O(N^2)\) communication costs, protecting client privacy. - **Momentum Mechanism**: Introduce a momentum mechanism (PGFedMo) to further enhance model training efficiency and performance. ### Conclusion By proposing the PGFed framework, this paper addresses the performance issues of traditional federated learning when dealing with heterogeneous data. Through explicit personalization and efficient communication mechanisms, it significantly improves the model's adaptability and generalization capabilities.

PGFed: Personalize Each Client's Global Objective for Federated Learning

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Pfedgf: Enabling Personalized Federated Learning Via Gradient Fusion

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Decentralized Directed Collaboration for Personalized Federated Learning

Efficient Personalized Federated Learning via Sparse Model-Adaptation

Personalized Federated Learning with First Order Model Optimization

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Learn What You Need in Personalized Federated Learning

FedL2P: Federated Learning to Personalize

Feed: Towards Personalization-Effective Federated Learning

A Unified Personalized Federated Learning Framework Ensuring Domain Generalization

Personalized Federated Learning under Mixture of Distributions

Personalized Federated Learning with Local Optimization Models

Personalized federated learning based on feature fusion

Personalized Federated Learning with Domain Generalization

FedCR: Personalized Federated Learning Based on Across-Client Common Representation with Conditional Mutual Information Regularization.

Improving Global Generalization and Local Personalization for Federated Learning

pFedKT: Personalized Federated Learning via Knowledge Transfer

Personalized Federated Learning with Multi-View Geometry Structure