Abstract:In the framework of real Hilbert spaces, we investigate first-order dynamical systems governed by monotone and continuous operators. We demonstrate that when the monotone operator flow is augmented with a Tikhonov regularization term, the resulting trajectory converges strongly to the element of the set of zeros with minimal norm. In addition, rates of convergence in norm for the trajectory's velocity and the operator along the trajectory can be derived in terms of the regularization function. In some particular cases, these rates of convergence can outperform the ones of the coercive operator flows and can be as fast as $O(\frac{1}{t})$ as $t \rightarrow +\infty$. In this way, we emphasize a surprising acceleration feature of the Tikhonov regularization. Additionally, we explore these properties for monotone operator flows that incorporate time rescaling and an anchor point and show that they are closely linked to second-order dynamics with a vanishing damping term. The convergence and convergence rate results we achieve for these systems complement recent findings for the Fast Optimistic Gradient Descent Ascent (OGDA) dynamics. When the monotone operator is defined as the identity minus a nonexpansive operator, the monotone equations transform into a fixed point problem. In such cases, explicitly discretizing the system with Tikhonov regularization, enhanced by an anchor point, leads to the Halpern fixed point iteration. We identify two regimes for the regularization sequence which ensure that the generated sequence of iterates converges strongly to the fixed point nearest to the anchor point. Furthermore, we establish a general theoretical framework that provides convergence rates for the vanishing of the discrete velocity and the fixed point residual. For certain regularization sequences, we derive specific convergence rates that align with those observed in continuous time.

An Adaptive Douglas-Rachford Dynamic System for Finding the Zero of Sum of Two Operators

Third Order Dynamical Systems for the Sum of Two Generalized Monotone Operators

Convergence Rates of Forward–Douglas–Rachford Splitting Method

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

Fast second-order dynamics with slow vanishing damping approaching the zeros of a monotone and continuous operator

Convergence of the Preconditioned Proximal Point Method and Douglas-Rachford Splitting in the Absence of Monotonicity

The splitting algorithms by Ryu and by Malitsky-Tam applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Approximating Iterations for Nonexpansive and Maximal Monotone Operators

The splitting algorithms by Ryu, by Malitsky-Tam, and by Campoy applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

A second order dynamical system method for solving a maximal comonotone inclusion problem

Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities

On a Stochastic Differential Equation with Correction Term Governed by a Monotone and Lipschitz Continuous Operator

An adaptive splitting algorithm for the sum of three operators

Doubly relaxed forward-Douglas--Rachford splitting for the sum of two nonconvex and a DC function

Three-Operator Splitting Method with Two-Step Inertial Extrapolation

Accelerated forward-backward and Douglas-Rachford splitting dynamics

Algorithm 1 : Non-stationary Forward – Douglas – Rachford splitting

Three-operator reflected forward-backward splitting algorithm with double inertial effects

An Inertial Parametric Douglas–Rachford Splitting Method for Nonconvex Problems

Tikhonov regularization of monotone operator flows not only ensures strong convergence of the trajectories but also speeds up the vanishing of the residuals

Strong convergence and fast rates for systems with Tikhonov regularization