Abstract:In practical federated learning (FL) systems, the presence of malicious Byzantine attacks and data heterogeneity often introduces biases into the learning process. However, existing Byzantine-robust methods typically only achieve a compromise between adaptability to different loss function types (including both strongly convex and non-convex) and robustness to heterogeneous datasets, but with non-zero optimality gap. Moreover, this compromise often comes at the cost of high computational complexity for aggregation, which significantly slows down the training speed. To address this challenge, we propose a federated learning approach called Federated Normalized Gradients Algorithm (Fed-NGA). Fed-NGA simply normalizes the uploaded local gradients to be unit vectors before aggregation, achieving a time complexity of $\mathcal{O}(pM)$, where $p$ represents the dimension of model parameters and $M$ is the number of participating clients. This complexity scale achieves the best level among all the existing Byzantine-robust methods. Furthermore, through rigorous proof, we demonstrate that Fed-NGA transcends the trade-off between adaptability to loss function type and data heterogeneity and the limitation of non-zero optimality gap in existing literature. Specifically, Fed-NGA can adapt to both non-convex loss functions and non-IID datasets simultaneously, with zero optimality gap at a rate of $\mathcal{O} (1/T^{\frac{1}{2} - \delta})$, where T is the iteration number and $\delta \in (0,\frac{1}{2})$. In cases where the loss function is strongly convex, the zero optimality gap achieving rate can be improved to be linear. Experimental results provide evidence of the superiority of our proposed Fed-NGA on time complexity and convergence performance over baseline methods.

Byzantine-resilient Federated Learning via Gradient Memorization

Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity

GAIN: Enhancing Byzantine Robustness in Federated Learning with Gradient Decomposition

Byzantine-resilient Federated Learning Employing Normalized Gradients on Non-IID Datasets

Split Aggregation: Lightweight Privacy-Preserving Federated Learning Resistant to Byzantine Attacks

Advancing Hybrid Defense for Byzantine Attacks in Federated Learning

Byzantine-robust Federated Learning Through Collaborative Malicious Gradient Filtering

Secure Federated Learning with Model Compression.

Gradients Stand-in for Defending Deep Leakage in Federated Learning

Gradient Leakage Attacks in Federated Learning: Research Frontiers, Taxonomy, and Future Directions

FL-AGN: A Privacy-Enhanced Federated Learning Method Based on Adaptive Gaussian Noise for Resisting Gradient Inference Attacks

A New Federated Learning Framework Against Gradient Inversion Attacks

AgrAmplifier: Defending Federated Learning Against Poisoning Attacks Through Local Update Amplification

Practical Differentially Private and Byzantine-resilient Federated Learning

Federated Learning Robust to Byzantine Attacks: Achieving Zero Optimality Gap

A Four-Pronged Defense Against Byzantine Attacks in Federated Learning

A Framework for Evaluating Gradient Leakage Attacks in Federated Learning

Byzantine-Robust Learning on Heterogeneous Data Via Gradient Splitting

Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learning

Shield Against Gradient Leakage Attacks: Adaptive Privacy-Preserving Federated Learning

Byzantine Fault-Tolerant Federated Learning Based on Trustworthy Data and Historical Information