Abstract:One of the most common defense strategies against model poisoning in federated learning is to employ a robust aggregator mechanism that makes the training more resilient. Many of the existing Byzantine robust aggregators provide theoretical guarantees and are empirically effective against certain categories of attacks. However, we observe that certain high-strength attacks can subvert the aggregator and collapse the training. In addition, most aggregators require identifying tolerant settings to converge. Impact of attacks becomes more pronounced when the number of Byzantines is near-majority, and becomes harder to evade if the attacker is omniscient with access to data, honest updates and aggregation methods. Motivated by these observations, we develop a robust aggregator called FedRISE for cross-silo FL that is consistent and less susceptible to poisoning updates by an omniscient attacker. The proposed method explicitly determines the optimal direction of each gradient through a sign-voting strategy that uses variance-reduced sparse gradients. We argue that vote weighting based on the cosine similarity of raw gradients is misleading, and we introduce a sign-based gradient valuation function that ignores the gradient magnitude. We compare our method against 8 robust aggregators under 6 poisoning attacks on 3 datasets and architectures. Our results show that existing robust aggregators collapse for at least some attacks under severe settings, while FedRISE demonstrates better robustness because of a stringent gradient inclusion formulation.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the issue of significant performance degradation or even complete failure of training due to model poisoning attacks in Federated Learning (FL), particularly in cross-silo settings. Specifically, the paper focuses on how to design a robust aggregator that can effectively aggregate gradients in the presence of a large number of Byzantine nodes, preventing malicious updates from disrupting the overall training process. ### Background and Motivation 1. **Challenges of Federated Learning**: - Federated Learning solves the collaboration problem between data silos by sharing statistical updates or models instead of actual data, thereby improving model quality while protecting privacy. - However, Federated Learning systems are vulnerable to various attacks, especially model poisoning attacks, which can be initiated by malicious nodes that send incorrect or carefully crafted gradient updates to disrupt the entire training process. 2. **Limitations of Existing Methods**: - **Median-based methods** (e.g., Krum, geometric median) and **outlier suppression methods** (e.g., trimmed mean) usually consider only a small subset of clients, ignoring data heterogeneity and being easily exploited by malicious nodes. - **Variance reduction methods** (e.g., trimmed mean, center clipping) can reduce the impact of extreme malicious updates but still cannot completely prevent directional interference. - **Clustering and anomaly detection methods** (e.g., CopodDOS, SignGuard) rely on metrics like L2 norm or cosine similarity, which may be unreliable in high-dimensional spaces. - **Multi-aggregator frameworks** (e.g., Detox, Robust-Tailor) provide multi-layer defense strategies but still require complex hyperparameter tuning. ### Main Contributions of the Paper 1. **Proposing FedRISE**: - FedRISE is a new robust aggregation function that determines the sign of each gradient using variance-reduced sparse gradients. - During the sign election process, each client's weight (rating) is based on the quality of its gradient, with the weight being independent of the previous round's computation results. 2. **Experimental Validation**: - The paper compares the performance of FedRISE with eight other robust aggregators across multiple datasets and architectures against six different poisoning attacks. - Experimental results show that existing robust aggregators can completely fail under certain attacks, while FedRISE demonstrates better robustness, especially under high-intensity attacks. 3. **Method Characteristics**: - **Gradient Clipping and Limiting**: Reduces the impact of malicious updates by limiting the L2 norm of gradients and constraining the size of gradients per coordinate. - **Gradient Sparsification**: Selects important gradients, removing noise and irrelevant updates to improve aggregation accuracy. - **Gradient Scoring**: Evaluates the quality of each client's gradient by calculating the dot product of gradient signs, ignoring the magnitude of gradients. - **Sign Election**: Determines the optimal direction of each gradient through weighted voting based on gradient scores. ### Conclusion By proposing FedRISE, the paper effectively addresses the issue of model poisoning attacks in Federated Learning, particularly in cross-silo settings. FedRISE better resists interference from malicious nodes, ensuring the stability and robustness of the training process.

FedRISE: Rating Induced Sign Election of Gradients for Byzantine Tolerant Federated Aggregation

FedRecAttack: Model Poisoning Attack to Federated Recommendation

Byzantine-robust Federated Learning Through Collaborative Malicious Gradient Filtering

EVFL: Towards Efficient Verifiable Federated Learning Via Parameter Reuse and Adaptive Sparsification

AgrAmplifier: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification

FedRDF: A Robust and Dynamic Aggregation Function against Poisoning Attacks in Federated Learning

Robust Federated Recommendation System

GAIN: Enhancing Byzantine Robustness in Federated Learning with Gradient Decomposition

Romoa: Robust Model Aggregation for the Resistance of Federated Learning to Model Poisoning Attacks

RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregation

Byzantine-resilient Federated Learning Employing Normalized Gradients on Non-IID Datasets

Split Aggregation: Lightweight Privacy-Preserving Federated Learning Resistant to Byzantine Attacks

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity

Secure Model Aggregation Against Poisoning Attacks for Cross-Silo Federated Learning With Robustness and Fairness

Byzantine-Robust Learning on Heterogeneous Data Via Gradient Splitting

Byzantine-resilient Federated Learning via Gradient Memorization

Robust federated learning with voting and scaling

RoseAgg: Robust Defense Against Targeted Collusion Attacks in Federated Learning

MANDERA: Malicious Node Detection in Federated Learning via Ranking

FedTruth: Byzantine-Robust and Backdoor-Resilient Federated Learning Framework