Abstract:This work proposes a general learned proximal alternating minimization algorithm, LPAM, for solving learnable two-block nonsmooth and nonconvex optimization problems. We tackle the nonsmoothness by an appropriate smoothing technique with automatic diminishing smoothing effect. For smoothed nonconvex problems we modify the proximal alternating linearized minimization (PALM) scheme by incorporating the residual learning architecture, which has proven to be highly effective in deep network training, and employing the block coordinate decent (BCD) iterates as a safeguard for the convergence of the algorithm. We prove that there is a subsequence of the iterates generated by LPAM, which has at least one accumulation point and each accumulation point is a Clarke stationary point. Our method is widely applicable as one can employ various learning problems formulated as two-block optimizations, and is also easy to be extended for solving multi-block nonsmooth and nonconvex optimization problems. The network, whose architecture follows the LPAM exactly, namely LPAM-net, inherits the convergence properties of the algorithm to make the network interpretable. As an example application of LPAM-net, we present the numerical and theoretical results on the application of LPAM-net for joint multi-modal MRI reconstruction with significantly under-sampled k-space data. The experimental results indicate the proposed LPAM-net is parameter-efficient and has favourable performance in comparison with some state-of-the-art methods.

Learning Low-Complexity Autoregressive Models via Proximal Alternating Minimization

Sparse Output Feedback Synthesis via Proximal Alternating Linearization Method

A Learned Proximal Alternating Minimization Algorithm and Its Induced Network for a Class of Two-block Nonconvex and Nonsmooth Optimization

The Convergence Properties of Infeasible Inexact Proximal Alternating Linearized Minimization

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation.

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation

Stochastic Gauss–Seidel type inertial proximal alternating linearized minimization and its application to proximal neural networks

Fitting ARMA Time Series Models without Identification: A Proximal Approach

On Partial Smoothness, Activity Identification and Faster Algorithms of L1 over L2 Minimization

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

A Proximal-Proximal Majorization-Minimization Algorithm for Nonconvex Rank Regression Problems.

Adaptive Accelerated Proximal Gradient Algorithm for Auto-Regressive Exogenous Models with Outliers

Proximal Quasi-Newton for Computationally Intensive L1-regularized M-estimators

Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation

Asynchronous Stochastic Proximal Optimization Algorithms with Variance Reduction

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Proximal Iteration for Nonlinear Adaptive Lasso

Regularized Q-Learning with Linear Function Approximation

Stochastic Proximal Gradient Algorithm with Minibatches. Application to Large Scale Learning Models

Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning

Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting