Abstract:This work considers two related learning problems in a federated attack prone setting: federated principal components analysis (PCA) and federated low rank column-wise sensing (LRCS). The node attacks are assumed to be Byzantine which means that the attackers are omniscient and can collude. We introduce a novel provably Byzantine-resilient communication-efficient and sampleefficient algorithm, called Subspace-Median, that solves the PCA problem and is a key part of the solution for the LRCS problem. We also study the most natural Byzantine-resilient solution for federated PCA, a geometric median based modification of the federated power method, and explain why it is not useful. Our second main contribution is a complete alternating gradient descent (GD) and minimization (altGDmin) algorithm for Byzantine-resilient horizontally federated LRCS and sample and communication complexity guarantees for it. Extensive simulation experiments are used to corroborate our theoretical guarantees. The ideas that we develop for LRCS are easily extendable to other LR recovery problems as well.

What problem does this paper attempt to address?

The main problem this paper attempts to address is the issue of Federated Principal Component Analysis (PCA) and Low Rank Column-wise Sensing (LRCS) in a Byzantine environment. Specifically: 1. **Federated PCA in a Byzantine Environment**: When multiple nodes collaborate to perform PCA calculations, some nodes may be subject to Byzantine attacks (i.e., malicious nodes can fully understand the system state and perform coordinated attacks). The paper proposes a new algorithm called "Subspace Median" that achieves Byzantine robustness in terms of communication and sample efficiency. Compared to traditional geometric median methods, this approach is more suitable for handling subspace estimation problems and performs better in cases of uneven data distribution. 2. **Horizontal Federated LRCS in a Byzantine Environment**: The paper also explores how to perform low-rank matrix recovery in a horizontal federated setting under Byzantine attacks. The authors propose an improved version of the alternating gradient descent and minimization (altGDmin) algorithm and demonstrate its advantages in terms of sample complexity and communication complexity. 3. **Algorithm Contributions**: The paper introduces an algorithm called "Subspace Median" for reliably estimating the principal subspace of high-dimensional data under Byzantine attacks. Additionally, it examines the limitations of the geometric median method in PCA problems and explains why it is not suitable for solving such issues. By introducing the new Subspace Median algorithm, the paper not only addresses the PCA problem but also provides an effective initialization step for the LRCS problem. In summary, this paper is primarily dedicated to developing efficient algorithms that can withstand Byzantine attacks to solve PCA and LRCS problems in federated learning scenarios. The effectiveness and robustness of the proposed methods are validated through theoretical analysis and extensive simulation experiments.

Byzantine-Resilient Federated PCA and Low Rank Column-wise Sensing

Resilient Two-Time-Scale Local Stochastic Gradient Descent for Byzantine Federated Learning

Practical Differentially Private and Byzantine-resilient Federated Learning

Resilient Federated Learning under Byzantine Attack in Distributed Nonconvex Optimization with 2-f Redundancy

Byzantine Attacker Identification in Collaborative Spectrum Sensing: A Robust Defense Framework

Seeking Consensus on Subspaces in Federated Principal Component Analysis

Robust Federated Recommendation System

Dual-domain Defenses for Byzantine-resilient Decentralized Resource Allocation

A Four-Pronged Defense Against Byzantine Attacks in Federated Learning

Byzantine-resilient Decentralized Stochastic Gradient Descent

SF-CABD: Secure Byzantine fault tolerance federated learning on Non-IID data

Byzantine-Resilient Federated Learning at Edge

Reconciling Privacy and Byzantine-robustness in Federated Learning

Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learning

Efficient Byzantine-Robust and Privacy-Preserving Federated Learning on Compressive Domain

Robust Federated Learning in a Heterogeneous Environment.

Scalable and Privacy-Preserving Federated Principal Component Analysis

Robust Federated Learning: Maximum Correntropy Aggregation Against Byzantine Attacks

Byzantine-Resilient Decentralized Collaborative Learning.

Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness

Fast Decentralized Federated Low Rank Matrix Recovery from Column-wise Linear Projections