Abstract:This paper deals with a second-order primal-dual dynamical system with Hessian-driven damping and Tikhonov regularization terms in connection with a convex-concave bilinear saddle point problem. We first obtain a fast convergence rate of the primal-dual gap along the trajectory generated by the dynamical system, and provide some integral estimates. Then, based on the setting of the parameters involved, we demonstrate that both the convergence rate of the primal-dual gap and the strong convergence of the trajectory can be achieved simultaneously. Furthermore, we evaluate the performance of the proposed system using two numerical examples.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to use the second - order primal - dual dynamical system with Hessian - driven damping and Tikhonov regularization to efficiently solve convex - concave bilinear saddle - point problems, and ensure the strong convergence of the minimum - norm solution of the saddle - point set and the fast convergence rate of the primal - dual gap**.
Specifically, the goals of the paper include:
1. **Propose a new dynamical system model**: Introduce a second - order primal - dual dynamical system (formula (11)) that includes slow viscous damping, extrapolation, Hessian - driven damping, and time - scaling, and combine it with Tikhonov regularization to solve convex - concave saddle - point problems.
2. **Prove fast convergence**: Under mild parameter assumptions, prove that the primal - dual gap on the trajectory generated by this dynamical system has a fast convergence rate of \( O\left(\frac{1}{t^{2q + s}}\right) \).
3. **Ensure strong convergence**: By appropriately adjusting the parameters, prove that there is a single mechanism such that the convergence rate of the primal - dual gap and the strong convergence of the trajectory to the minimum - norm element of the saddle - point set can be achieved simultaneously.
4. **Verify by numerical experiments**: Through numerical experiments, it is shown that the proposed dynamical system not only significantly suppresses the oscillations that occur during the iteration process, but may also accelerate the convergence of values.
### Summary of Mathematical Formulas
- **Objective function**:
\[
L(x, y)=f(x)+\langle Kx, y\rangle - g(y)
\]
where \( f: \mathbb{R}^n\rightarrow\mathbb{R} \) and \( g: \mathbb{R}^m\rightarrow\mathbb{R} \) are twice continuously differentiable convex functions, and \( K: \mathbb{R}^n\rightarrow\mathbb{R}^m \) is a continuous linear operator.
- **Augmented Lagrangian function**:
\[
L_t(x, y)=L(x, y)+\frac{c}{2t^p}(\|x\|^2-\|y\|^2)
\]
- **Dynamical system equations**:
\[
\begin{cases}
\ddot{x}(t)+\frac{\alpha}{t^q}\dot{x}(t)+\gamma\frac{d}{dt}(\nabla_xL_t(x(t), y(t)+\theta(t)\dot{y}(t)))+t^s\nabla_xL_t(x(t), y(t)+\theta(t)\dot{y}(t)) = 0,\\
\ddot{y}(t)+\frac{\alpha}{t^q}\dot{y}(t)-\gamma\frac{d}{dt}(\nabla_yL_t(x(t)+\theta(t)\dot{x}(t), y(t)))-t^s\nabla_yL_t(x(t)+\theta(t)\dot{x}(t), y(t)) = 0,
\end{cases}
\]
where \( \alpha>1 \), \( 0 < q < 1 \), \( \gamma>0 \), \( s>0 \), and \( \theta(t)=\frac{t^{2q + s-2}\gamma qt^{2q-1}+\gamma t^q}{(\alpha - 1)(t^q + s-\gamma qt^{q-1})} \).
### Conclusion
Through the above methods, the paper successfully solves the efficient solution of convex - concave saddle - point problems and ensures strong convergence and fast convergence rates. This provides new ideas and tools for the research of optimization algorithms.