Abstract:Low Rank Regularization (LRR), in essence, involves introducing a low rank or approximately low rank assumption to target we aim to learn, which has achieved great success in many data analysis tasks. Over the last decade, much progress has been made in theories and applications. Nevertheless, the intersection between these two lines is rare. In order to construct a bridge between practical applications and theoretical studies, in this paper we provide a comprehensive survey for LRR. Specifically, we first review the recent advances in two issues that all LRR models are faced with: <math>(1)</math> rank-norm relaxation, which seeks to find a relaxation to replace the rank minimization problem; <math>(2)</math> model optimization, which seeks to use an efficient optimization algorithm to solve the relaxed LRR models. For the first issue, we provide a detailed summarization for various relaxation functions and conclude that the non-convex relaxations can alleviate the punishment bias problem compared with the convex relaxations. For the second issue, we summary the representative optimization algorithms used in previous studies, and analysis their advantages and disadvantages. As the main goal of this paper is to promote the application of non-convex relaxations, we conduct extensive experiments to compare different relaxation functions. The experimental results demonstrate that the non-convex relaxations generally provide a large advantage over the convex relaxations. Such a result is inspiring for further improving the performance of existing LRR models.

Optimality analysis on l0/1 -sparsity regularized nonsmooth optimization

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

A Sufficient Condition on Monotonic Increase of the Number of Nonzero Entry in the Optimizer of L1 Norm Penalized Least-Square Problem

Penalty Decomposition Methods for $L0$-Norm Minimization

Avoiding strict saddle points of nonconvex regularized problems

Neural Network for a Class of Sparse Optimization with L0-regularization.

First-order Optimality Conditions for Two Classes of Generalized Nonsmooth Semi-Infinite Optimization

Tikhonov Regularization with L^0-Term Complementing a Convex Penalty: L^1 Convergence under Sparsity Constraints

Revisiting $L_q(0\leq q<1)$ Norm Regularized Optimization

Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit

Piecewise Linear Relationship Between L1 Norm Objective Functional Values and L∞ Norm Constraint Specifications

First- and second-order optimality conditions of nonsmooth sparsity multiobjective optimization via variational analysis

A Simple Neural Network for Sparse Optimization with $l_1$ Regularization

Study on L_1 over L_2 Minimization for Nonnegative Signal Recovery

Low Rank Regularization: A review

Optimality conditions for the nonlinear programming problems on Riemannian manifolds

Study on <i>L</i><sub>1</sub> over <i>L</i><sub>2</sub> Minimization for Nonnegative Signal Recovery

Optimality and Complexity for Constrained Optimization Problems with Nonconvex Regularization.

Multi-parameter Tikhonov Regularization with the ℓ 0 Sparsity Constraint