Parameter-free proximal bundle methods with adaptive stepsizes for hybrid convex composite optimization problems

Renato D.C. Monteiro,Honghao Zhang

2024-10-28

Abstract:This paper develops a parameter-free adaptive proximal bundle method with two important features: 1) adaptive choice of variable prox stepsizes that "closely fits" the instance under consideration; and 2) adaptive criterion for making the occurrence of serious steps easier. Computational experiments show that our method performs substantially fewer consecutive null steps (i.e., a shorter cycle) while maintaining the number of serious steps under control. As a result, our method performs significantly less number of iterations than its counterparts based on a constant prox stepsize choice and a non-adaptive cycle termination criterion. Moreover, our method is very robust relative to the user-provided initial stepsize.

Optimization and Control

What problem does this paper attempt to address?

This paper attempts to address the efficiency and adaptability issues of existing proximal bundle methods in the Hybrid Convex Composite Optimization (HCCO). Specifically, the article proposes an Adaptive Generalized Proximal Bundle (Ad - GPB), whose main objective is to improve existing methods through the following two key features:

1. **Adaptive selection of variable proximal step - sizes**: Select variable proximal step - sizes (prox step - sizes) in a "closely - fitted" manner according to specific instance situations, so as to better adapt to different optimization problem instances.
2. **Adaptive criteria make serious steps more likely to occur**: Introduce adaptive criteria to make serious steps more likely to occur, thereby reducing the number of consecutive null steps.

These improvements aim to significantly reduce the number of iterations, improve computational performance, and be more robust with respect to the initial step - sizes provided by the user. Through these improvements, the Ad - GPB method reduces the amount of computation required per iteration while maintaining theoretical guarantees, especially when dealing with complex HCCO problems.

### Summary of Mathematical Expressions

- Objective function:
\[
\phi^* := \min\{\phi(x) := f(x)+h(x) : x\in\mathbb{R}^n\}
\]
where \(f, h:\mathbb{R}^n\rightarrow\mathbb{R}\cup\{+\infty\}\) are proper lower - semicontinuous convex functions and satisfy \(\text{dom }h\subseteq\text{dom }f\).

- Proximal sub - problem:
\[
x_j=\arg\min_{u\in\mathbb{R}^n}\left\{\Gamma_j(u)+\frac{1}{2\lambda_j}\|u - x_c\|^2\right\}
\]
where \(\Gamma_j\) is a simple convex function approximation of \(\phi\), and \(x_c\) is the current proximal center.

- Adaptive criteria:
\[
t_j\leq\beta_k[\phi(y_j)-\hat{n}_k]+\frac{\varepsilon}{4}
\]
where \(t_j = \phi(y_j)-m_j\), and \(m_j\) is the optimal value of the sub - problem.

Through these improvements, the Ad - GPB method can find a solution close to the optimal solution in fewer iterations, thereby improving the efficiency and robustness of solving the hybrid convex composite optimization problem.