Abstract:In this article, we propose a novel solution for nonconvex problems of multiple variables, especially for those typically solved by an alternating minimization (AM) strategy that splits the original optimization problem into a set of subproblems corresponding to each variable and then iteratively optimizes each subproblem using a fixed updating rule. However, due to the intrinsic nonconvexity of the original optimization problem, the optimization can be trapped into a spurious local minimum even when each subproblem can be optimally solved at each iteration. Meanwhile, learning-based approaches, such as deep unfolding algorithms, have gained popularity for nonconvex optimization; however, they are highly limited by the availability of labeled data and insufficient explainability. To tackle these issues, we propose a meta-learning based alternating minimization (MLAM) method that aims to minimize a part of the global losses over iterations instead of carrying minimization on each subproblem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance. The proposed MLAM maintains the original algorithmic principle, providing certain interpretability. We evaluate the proposed method on two representative problems, namely, bilinear inverse problem: matrix completion and nonlinear problem: Gaussian mixture models. The experimental results validate the proposed approach outperforms AM-based methods.

Variants of Alternating Minimization Method with Sublinear Rates of Convergence for Convex Optimization

Little-$o$ Convergence Rates for Several Alternating Minimization Methods

A Proximal Alternating Linearization Method for Nonconvex Optimization Problems

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

A Class of Alternating Linearization Algorithms for Nonsmooth Convex Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

AN APPROXIMATE ALTERNATING LINEARIZATION DECOMPOSITION METHOD

Meta-learning Based Alternating Minimization Algorithm for Non-convex Optimization

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming.

A Preconditioned Alternating Minimization Framework for Nonconvex and Half Quadratic Optimization.

A Proximal Parallel Splitting Method for Minimizing Sum of Convex Functions with Linear Constraints.

Majorized Semi-proximal Alternating Coordinate Method for Nonsmooth Convex-Concave Minimax Optimization

An Alternating Linearization Bundle Method for a Class of Nonconvex Optimization Problem with Inexact Information

Linear Convergence Rate of Splitting Algorithms for Multi-Block Constrained Convex Minimizations

A General Proximal Alternating Minimization Method with Application to Nonconvex Nonsmooth 1D Total Variation Denoising

Semi-Proximal Point Method for Nonsmooth Convex-Concave Minimax Optimization

Inexact Alternating-Direction-Based Contraction Methods for Separable Linearly Constrained Convex Optimization

Improved Algorithms for Convex-Concave Minimax Optimization

Variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities