Abstract:This work develops a provably accurate fully-decentralized alternating projected gradient descent (GD) algorithm for recovering a low rank (LR) matrix from mutually independent projections of each of its columns, in a fast and communication-efficient fashion. To our best knowledge, this work is the first attempt to develop a provably correct decentralized algorithm (i) for any problem involving the use of an alternating projected GD algorithm; (ii) and for any problem in which the constraint set to be projected to is a non-convex set.

What problem does this paper attempt to address?

### The Problem Addressed by This Paper This paper aims to address the problem of recovering low-rank matrices from the linear projections of column vectors (Low Rank Column-wise Compressive Sensing, LRcCS) and develops an alternating gradient descent and minimization algorithm (AltGDmin) in a decentralized federated setting. Specifically: 1. **Research Background**: - Data is federated without a central coordinating node. - The goal is to recover an unknown low-rank matrix \(X^* = [x_1^*, x_2^*, \ldots, x_q^*]\) from the linear projections of each column vector. - Different nodes obtain different subsets \(y_k\), where \(y_k\) is a vector of length \(m\) and \(m < n\). 2. **Main Issues**: - Recovering the low-rank matrix \(X^*\) in a decentralized federated environment. - Providing an algorithm and its theoretical guarantees to ensure that the required sample complexity and iteration complexity are as low as possible. 3. **Application Scenarios**: - Distributed/Federated sketching, especially on mobile devices or IoT devices. - Dynamic MRI or MRI processing of multiple patient images. - Multi-task learning (including supervised learning, Bandit learning, and reinforcement learning). 4. **Main Contributions**: - Developed a decentralized alternating gradient descent and minimization algorithm (Dec-AltGDmin). - Provided non-asymptotic theoretical guarantees, demonstrating the required sample complexity and iteration complexity, and proved that the algorithm is fast and communication-efficient. - Validated the algorithm's performance through extensive simulations and compared it with other benchmark algorithms. In summary, this paper aims to develop an algorithm suitable for low-rank matrix recovery in a decentralized federated environment, providing rigorous theoretical analysis and experimental validation.

Fast Decentralized Federated Low Rank Matrix Recovery from Column-wise Linear Projections

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

Efficient Federated Low Rank Matrix Completion

Scalable Algorithms for Locally Low-Rank Matrix Modeling

Low-rank matrix recovery with non-quadratic loss: projected gradient method and regularity projection oracle

A Preconditioned Riemannian Gradient Descent Algorithm for Low-Rank Matrix Recovery

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Decentralized Low-Rank Matrix Completion

Fast Algorithms for Recovering a Corrupted Low-Rank Matrix

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Fast Gradient Method for Low-Rank Matrix Estimation

Improved Algorithms for Matrix Recovery from Rank-One Projections

Predictive Low Rank Matrix Learning under Partial Observations: Mixed-Projection ADMM

Efficient Low Rank Matrix Recovery With Flexible Group Sparse Regularization

A Fast Algorithm for Low Rank + Sparse column-wise Compressive Sensing

Fast Convex Optimization Algorithms for the Recovery of a Corrupted Low-Rank Matrix

The Global Geometry of Centralized and Distributed Low-rank Matrix Recovery without Regularization

Nonconvex Deterministic Matrix Completion by Projected Gradient Descent Methods

Convergence of projected Landweber iteration for matrix rank minimization

Fast Algorithms for Recovering a Corrupted