Abstract:Low-rank matrix factorization (LRMF) is a powerful approach that can recover useful information with low-rank features from corrupted data and has been broadly applied in various scenarios such as computer vision and statistics. Existing convex relaxation-based LRMF models' performance and solutions are often limited by unsatisfied recovery accuracy and prolonged execution time due to excessive relaxation bias and heavy computational burden. In this paper, we first explore one class of parameterized non-convex penalty function and generalize it as a non-convex relaxation substitution term for the original rank function in the LRMF, which can effectively enhance the recovery capability for low-rank data. In order to reduce the computational burden of the nuclear-norm-based optimization scheme and make LRMF applicable to large-scale data. We propose to perform bilinear decomposition on the low-rank matrix and derive the bilinear factorization formula under the parameterized non-convex alternative. With the above considering, a low-rank matrix recovery model via bilinear decomposition and non-convex regularization, denoted as NCBF, is formulated. Further, the optimal solution algorithm for the NCBF model is developed in the framework of the block coordinate descent, combining simultaneously three strategies of decoupling, proximal gradient descent, and block prox-linear. Also, the convergence condition for the proposed algorithm has been given. We perform extensive experiments using synthetic data and real-world image datasets. The synthetic data test results demonstrate the dual superiority of the bilinear decomposition scheme in terms of operation time and estimation accuracy. Application of denoising to both generic and CT image datasets, experimental results show that NCBF outperforms several existing methods under both subjective and objective metrics evaluations.

A Momentum Accelerated Algorithm for ReLU-Based Nonlinear Matrix Decomposition

A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix Decomposition

Accelerated Algorithms for Nonlinear Matrix Decomposition with the ReLU function

Improving Fault Tolerance for Reliable DNN Using Boundary-Aware Activation

Accelerated analysis on the triple momentum method for a two-layer ReLU neural network

Deep unrolling networks with recurrent momentum acceleration for nonlinear inverse problems

The effect of Target Normalization and Momentum on Dying ReLU

Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks

Stochastic Momentum Method with Double Acceleration for Regularized Empirical Risk Minimization

Sine Activated Low-Rank Matrices for Parameter Efficient Learning

Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training

Linearization of ReLU Activation Function for Neural Network-Embedded Optimization:Optimal Day-Ahead Energy Scheduling

Competition-based Adaptive ReLU for Deep Neural Networks

Optimization of ReLU Neural Networks using Quotient Stochastic Gradient Descent

A Momentum-Incorporated Non-Negative Latent Factorization of Tensors Model for Dynamic Network Representation

FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks

A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization

Low-rank matrix factorization with nonconvex regularization and bilinear decomposition

An Improved Reacceleration Optimization Algorithm Based on the Momentum Method for Image Recognition

EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks.