Abstract:In this article, we study inertial algorithms for numerically solving monotone inclusions involving the sum of a maximally monotone and a cocoercive operator. In particular, we analyze the convergence of inertial and relaxed versions of the nonlinear forward-backward with momentum (NFBM). We propose an inertial version including a relaxation step, and a second version considering a double-inertial step with additional momentum. By applying NFBM to specific monotone inclusions, we derive inertial and relaxed versions of algorithms such as forward-backward, forward-half-reflect-backward (FHRB), Chambolle-Pock, Condat-Vũ, among others, thereby recovering and extending previous results from the literature for solving monotone inclusions involving maximally monotone, cocoercive, monotone and Lipschitz, and linear bounded operators. We also present numerical experiments on image restoration, comparing the proposed inertial and relaxation algorithms. In particular, we compare the inertial FHRB with its non-inertial and momentum versions. Additionally, we compare the numerical convergence for larger step-sizes versus relaxation parameters and introduce a restart strategy that incorporates larger step-sizes and inertial steps to further enhance numerical convergence.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to study and analyze inertial algorithms for numerically solving monotone inclusions, especially those involving the sum of a maximally monotone operator and a conjugate operator. Specifically, the authors study the convergence of inertial and relaxed versions of the Nonlinear Forward - Backward with Momentum (NFBM) algorithm.
#### Specific problem description
In the Hilbert space \(H\), consider the following problem:
**Problem 2.1**: Let \(A: H\rightarrow 2^{H}\) be a maximally monotone operator and \(C: H\rightarrow H\) be a \(\mu\)-conjugate operator (\(\mu\in(0,+\infty)\)). The goal is to find \(x\in H\) such that:
\[0\in(A + C)x\]
Assume that its solution set \(Z\) is non - empty.
This problem can be solved by the NFBM algorithm. In each iteration of the NFBM algorithm, it involves one calculation of the conjugate operator \(C\) and one calculation of the distorted resolvent induced by a Lipschitz operator, which generalizes the classical Forward - Backward (FB) algorithm. Moreover, NFBM allows the use of variable metrics induced by self - adjoint strongly monotone linear operators.
#### Main contributions
The main contributions of this paper are the introduction of inertial and relaxation steps into the NFBM algorithm, and the proposal of two new inertial versions:
1. **Inertial NFBM with relaxation step**: Adds a relaxation step on the basis of the standard NFBM.
2. **NFBM with double inertial step - lengths**: Adds a second inertial step - length and introduces an additional momentum term.
For these two methods, the authors prove that their iterative sequences weakly converge to the solution points of the monotone inclusion problem. By applying these improved NFBM algorithms, inertial and relaxation versions of other classical algorithms (such as FB, FHRB, Chambolle - Pock, Condat - Vũ, etc.) can be derived, thus extending the results in the existing literature.
In addition, the authors also conduct numerical experiments on image inpainting, compare the performance of the proposed inertial and non - inertial algorithms, especially in the case of a fixed step - size, and show the acceleration effect of the new algorithms in the number of iterations. When the step - size is close to its allowed maximum, a restart strategy is introduced, which combines a larger step - size and an inertial step - length to further improve the numerical convergence speed.
### Summary
In general, this paper is dedicated to developing and analyzing improved inertial algorithms to accelerate the solution of monotone inclusion problems involving maximally monotone operators and conjugate operators. By introducing inertial and relaxation steps, it not only improves the performance of existing algorithms but also provides new theoretical tools and numerical methods for solving such problems.