Abstract:We present a numerical iterative optimization algorithm for the minimization of a cost function consisting of a linear combination of three convex terms, one of which is differentiable, a second one is prox-simple and the third one is the composition of a linear map and a prox-simple function. The algorithm's special feature lies in its ability to approximate, in a single iteration run, the minimizers of the cost function for many different values of the parameters determining the relative weight of the three terms in the cost function. A proof of convergence of the algorithm, based on an inexact variable metric approach, is also provided. As a special case, one recovers a generalization of the primal-dual algorithm of Chambolle and Pock, and also of the proximal-gradient algorithm. Finally, we show how it is related to a primal-dual iterative algorithm based on inexact proximal evaluations of the non-smooth terms of the cost function.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is in optimization problems, how to effectively calculate the minimum value of a cost function that contains three convex terms (one of which is differentiable, another is approximately simple, and the third is a combination of a linear mapping and an approximately simple function), especially when the values of parameters \(\lambda\) and \(\mu\) are uncertain or multiple different values need to be considered. Specifically, the paper proposes an iterative algorithm that can approximately calculate the minimum value of the cost function for many different parameter values in a single run. This problem is very common in image processing and other inverse problems, where the first term of the cost function usually corresponds to the data fidelity term (such as least squares), and the remaining two terms are added as penalty terms to exclude "non - physical" solutions due to the ill - posedness of the underlying inverse problem. The main contribution of the paper lies in introducing a new iterative algorithm. This algorithm can not only handle the above - mentioned optimization problems, but also approximately calculate the minimum value under different parameter values by introducing varying penalty parameters \(\lambda_n\) and \(\mu_n\). This algorithm uses these varying penalty parameters in each iteration instead of the fixed parameters \(\lambda\) and \(\mu\), thus providing a method of optimization continuation. In addition, the paper also provides a convergence proof of this algorithm and discusses its relationship with the primal - dual method of inexact approximate evaluation.

Convergence analysis of a primal-dual optimization-by-continuation algorithm

Convergence analysis of a primal–dual optimization-by-continuation algorithm

Convergence analysis of approximate primal solutions in dual first-order methods

Parametrization and convergence of a primal-dual block-coordinate approach to linearly-constrained nonsmooth optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Local Linear Convergence Analysis of Primal–Dual Splitting Methods

A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions

A splitting primal-dual proximity algorithm for solving composite optimization problems

A New Primal-Dual Algorithm for a Class of Nonlinear Compositional Convex Optimization Problems

A new primal-dual algorithm for structured convex optimization involving a lipschitzian term

A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization

Local Linear Convergence of a Primal-Dual Algorithm for the Augmented Convex Models

Convergence rate analysis of fast primal-dual methods with scalings for linearly constrained convex optimization problems

On the Linear Convergence of the General First Order Primal-Dual Algorithm

Some Primal-Dual Theory for Subgradient Methods for Strongly Convex Optimization

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

Primal-dual proximal bundle and conditional gradient methods for convex problems

Analysis of Primal-Dual Langevin Algorithms

Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization

A simple primal-dual algorithm for minimization of the sum of three convex functions

Automated Lyapunov Analysis of Primal-Dual Optimization Algorithms: An Interpolation Approach