Abstract:Federated Learning (FL) is a distributed machine learning scheme that enables clients to train a shared global model without exchanging local data. The presence of label noise can severely degrade the FL performance, and some existing studies have focused on algorithm design for label denoising. However, they ignored the important issue that clients may not apply costly label denoising strategies due to them being self-interested and having heterogeneous valuations on the FL performance. To fill this gap, we model the clients' interactions as a novel label denoising game and characterize its equilibrium. We also analyze the price of stability, which quantifies the difference in the system performance (e.g., global model accuracy, social welfare) between the equilibrium outcome and the socially optimal solution. We prove that the equilibrium outcome always leads to a lower global model accuracy than the socially optimal solution does. We further design an efficient algorithm to compute the socially optimal solution. Numerical experiments on MNIST dataset show that the price of stability increases as the clients' data become noisier, calling for an effective incentive mechanism.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: In quality - aware Federated Learning (FL), due to the existence of label noise, clients may choose not to perform costly label - denoising processing, which will lead to a decline in the overall performance of Federated Learning. Specifically: 1. **The impact of label noise on the performance of Federated Learning**: - Label noise can seriously affect the performance of Federated Learning, especially in the case of client - data heterogeneity and self - interested behavior. Existing research mainly focuses on algorithm design to remove label noise, but ignores the problem that clients may be unwilling to execute these denoising strategies due to their own interests and resource limitations. 2. **Clients' self - interested behavior and denoising strategies**: - Clients are self - interested in Federated Learning, and they have different valuations of Federated Learning performance. Therefore, how to decide the label - denoising strategy becomes a key issue. Clients need to find a balance between improving model performance and reducing denoising costs. 3. **Efficiency loss of system performance**: - To quantify this efficiency loss, the author introduced the "Price of Stability (PoS)", which measures the difference between the performance of the system in the Nash equilibrium state and the social optimal solution. Specifically, PoS measures the gap between global model accuracy and social welfare between Nash equilibrium and the optimal solution. ### Main contributions of the paper - **Study PoS in quality - aware Federated Learning for the first time**: By modeling the non - cooperative game between clients, analyzing its Nash equilibrium, and proving that the Nash equilibrium always leads to a higher average noise rate and lower global model accuracy. - **Propose an effective algorithm**: Designed an efficient algorithm to calculate the social optimal solution, and verified through experiments the impact of different types of label noise (random flipping and instance - dependent noise) on global model accuracy. - **Empirical analysis**: Numerical experiments show that as client data becomes more and more noisy, PoS will increase, thus emphasizing the need for effective incentive mechanisms to improve the overall efficiency of the system. ### Key formulas - Average noise rate: \[ \bar{x}(x)=\frac{\sum_{n \in N} d_{n}x_{n}}{\sum_{n' \in N} d_{n'}} \] - Global model accuracy: \[ A(x)=g(\bar{x}(x)) \] where \(g(\cdot)\) is a concave decreasing function. - Social optimal solution: \[ x^{\text{SWM}}=\arg \max_{x} SW(x)=\sum_{n \in N} P_{n}(x) \] - Nash equilibrium: \[ x^{\text{NE}}_{n}= \begin{cases} \epsilon_{n}&\text{if }h^{\text{NE}}_{n}(\epsilon_{n})>0\\ 0&\text{if }h^{\text{NE}}_{n}(0)<0\\ x^{\circ,\text{NE}}_{n}&\text{otherwise} \end{cases} \] Through these analyses, the paper reveals that in quality - aware Federated Learning, clients' self - interested behavior will lead to a significant decline in system performance and proposes improvement directions for future research.

Price of Stability in Quality-Aware Federated Learning

Federated Data Quality Assessment Approach: Robust Learning With Mixed Label Noise

Optimality and Stability in Federated Learning: A Game-theoretic Approach

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Collaboratively Learning Federated Models from Noisy Decentralized Data

Privacy-Preserving Federated Learning Against Label-Flipping Attacks on Non-IID Data

Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients

Regulating Clients' Noise Adding in Federated Learning without Verification

Federated Learning with Extremely Noisy Clients via Negative Distillation

Rethinking Client Drift in Federated Learning: A Logit Perspective

Overhead-free Noise-tolerant Federated Learning: A New Baseline

Federated Learning with Noisy User Feedback

Data Quality-Aware Client Selection in Heterogeneous Federated Learning

Age-of-Information-Aware Federated Learning

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

Understanding Generalization of Federated Learning via Stability: Heterogeneity Matters

Welfare and Fairness Dynamics in Federated Learning: A Client Selection Perspective

Federated Learning With Unreliable Clients: Performance Analysis and Mechanism Design

Revisiting Early-Learning Regularization When Federated Learning Meets Noisy Labels

A game-theoretic approach for federated learning: A trade-off among privacy, accuracy and energy

Tackling Noisy Clients in Federated Learning with End-to-end Label Correction