Noisy Low Rank Column-wise Sensing

Ankit Pratap Singh,Namrata Vaswani
2024-09-13
Abstract:This letter studies the AltGDmin algorithm for solving the noisy low rank column-wise sensing (LRCS) problem. Our sample complexity guarantee improves upon the best existing one by a factor $\max(r, \log(1/\epsilon))/r$ where $r$ is the rank of the unknown matrix and $\epsilon$ is the final desired accuracy. A second contribution of this work is a detailed comparison of guarantees from all work that studies the exact same mathematical problem as LRCS, but refers to it by different names.
Signal Processing,Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the Low Rank Column - wise Sensing (LRCS) problem, especially in the presence of noise. Specifically, the research objective is to recover a low - rank matrix \(X^*\) from the linear projections of each column vector, and these linear projections are contaminated by noise. This problem has important applications in areas such as federated sketching, accelerated dynamic MRI, and multi - task linear representation learning. ### Problem Background The low - rank matrix column - sensing problem can be formalized as follows: Given an \(n\times q\) low - rank matrix \(X^*\) with rank \(r\), we hope to recover \(X^*\) through \(m\) noisy linear measurements \(y_k = A_k x^*_k+v_k\), where \(A_k\) is a known measurement matrix and \(v_k\) is additive Gaussian white noise. ### Main Contributions 1. **Improved Sample Complexity**: - The paper proposes the AltGDmin algorithm and proves that its sample complexity is \(\frac{\max(r,\log(1 / \epsilon))}{r}\) times better than existing methods, where \(r\) is the rank of the matrix and \(\epsilon\) is the final desired accuracy. 2. **Detailed Comparison**: - The paper makes a detailed comparison of all works that study the same mathematical problem (although with different names), including "Multi - task Linear Representation Learning", "Federated Low - Rank Matrix Sketching", and "Low - Rank Phase Retrieval (LRPR)". ### Sample Complexity Improvement For the noisy case, the existing best sample complexity is \(O\left(\frac{n r^2\log n\max(1,\text{NSR}^2 / r,\log(1 / \epsilon),r\text{NSR}/\epsilon^2\log(1 / \epsilon))}{\epsilon^2}\right)\), while the AltGDmin algorithm proposed in this paper improves this complexity to \(O\left(\frac{n r\max(r,\log(1 / \epsilon))\log n}{\epsilon^2\text{NSR}}\right)\). ### Formula Summary - **NSR (Signal - to - Noise Ratio)**: \[ \text{NSR}:=\frac{q\sigma_v^2}{\sigma_r^{*2}} \] where \(q\) is the number of columns, \(\sigma_v^2\) is the noise variance, and \(\sigma_r^{*2}\) is the square of the smallest non - zero singular value of \(X^*\). - **Frobenius Norm**: \[ \|X\|_F=\sqrt{\sum_{i = 1}^n\sum_{j = 1}^q|x_{ij}|^2} \] - **Subspace Distance**: \[ \text{SD2}(U_1,U_2):=\|(I - U_1U_1^{\top})U_2\| \] Through these improvements, the AltGDmin algorithm shows higher efficiency and better performance when dealing with the noisy low - rank matrix column - sensing problem.