Abstract:The low-rank matrix completion has gained rapidly increasing attention from researchers in recent years for its efficient recovery of the matrix in various fields. Numerous studies have exploited the popular neural networks to yield low-rank outputs under the framework of low-rank matrix factorization. However, due to the discontinuity and nonconvexity of rank function, it is difficult to directly optimize the rank function via back propagation. Although a large number of studies have attempted to find relaxations of the rank function, e.g., Schatten- $p$ norm, they still face the following issues when updating parameters via back propagation: 1) These methods or surrogate functions are still non-differentiable, bringing obstacles to deriving the gradients of trainable variables. 2) Most of these surrogate functions perform singular value decomposition upon the original matrix at each iteration, which is time-consuming and blocks the propagation of gradients. To address these problems, in this paper, we develop an efficient block-wise model dubbed differentiable low-rank learning (DLRL) framework that adopts back propagation to optimize the Multi-Schatten- $p$ norm Surrogate (MSS) function. Distinct from the original optimization of this surrogate function, the proposed framework avoids singular value decomposition to admit the gradient propagation and builds a block-wise learning scheme to minimize values of Schatten- $p$ norms. Accordingly, it speeds up the computation and makes all parameters in the proposed framework learnable according to a predefined loss function. Finally, we conduct substantial experiments in terms of image recovery and collaborative filtering. The experimental results verify the superiority of the proposed framework in terms of both runtimes and learning performance compared with other state-of-the-art low-rank optimization methods. Our codes are available at https://github.com/chenzl23/DLRL .

Fast and Robust Low-Rank Learning over Networks: A Decentralized Matrix Quantile Regression Approach

Online Learning over Distributed Low-Rank Networks Via Sequential Power Iteration

Fast and Robust Sparsity Learning Over Networks: A Decentralized Surrogate Median Regression Approach

Fast Low-Rank Matrix Learning with Nonconvex Regularization

Distributed Online Quantile Regression over Networks with Quantized Communication.

Distributed Statistical Estimation in Quantile Regression over a Network

A Scalable, Adaptive and Sound Nonconvex Regularizer for Low-rank Matrix Learning

Computationally Efficient and Statistically Optimal Robust Low-rank Matrix and Tensor Estimation

Efficient and Differentiable Low-Rank Matrix Completion with Back Propagation

Communication-efficient Estimation of Quantile Matrix Regression for Massive Datasets

Computationally Efficient and Statistically Optimal Robust Low-rank Matrix Estimation

Sparse and Low-Rank Matrix Quantile Estimation with Application to Quadratic Regression

Robust reduced rank regression in a distributed setting

Large-Scale Affine Matrix Rank Minimization with a Novel Nonconvex Regularizer.

Large-scale low-rank matrix learning with nonconvex regularizers

Decentralized Low-Rank Matrix Completion

A Unified Computational and Statistical Framework for Nonconvex Low-rank Matrix Estimation

Fast Algorithms for Recovering a Corrupted Low-Rank Matrix

Robust low-rank training via approximate orthonormal constraints

Robust Recovery of Low-Rank Matrices Via Non-Convex Optimization

Efficient Low Rank Matrix Approximation Via Orthogonality Pursuit and ℓ2 Regularization