Abstract:We develop algorithms for the optimization of convex objectives that have Hölder continuous $q$-th derivatives with respect to a $p$-norm by using a $q$-th order oracle, for $p, q \geq 1$. We can also optimize other structured functions. We do this by developing a non-Euclidean inexact accelerated proximal point method that makes use of an inexact uniformly convex regularizer. We also provide nearly matching lower bounds for any deterministic algorithm that interacts with the function via a local oracle.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the challenges regarding Non - Euclidean High - Order Smooth Convex Optimization. Specifically, the researchers focus on how to efficiently optimize functions when the given functions have Hölder - continuous high - order derivatives. Such problems are very important in many practical applications, such as machine learning, signal processing, and control theory.
### Main Problem Description
1. **Optimization Objective**: The goal of the paper is to develop an algorithm for optimizing convex function \(f\) with Hölder - continuous \(q\)-th order derivatives, which are defined with respect to the \(p\)-norm. That is, for all \(x, y\in\mathbb{R}^d\), the following condition is satisfied:
\[
\|\nabla^q f(x)-\nabla^q f(y)\|_{p^*}\leq L\|x - y\|_p^{\nu}
\]
where \(p^*=(1 - 1/p)^{-1}\), \(q\in\mathbb{Z}_+\), \(\nu\in(0, 1]\). In this case, we call the function \(f\) \(q\)-th order \((L, \nu)\)-Hölder smooth.
2. **Optimization Method**: To achieve this goal, the author proposes a non - Euclidean inexact accelerated proximal point method, which uses an inexact uniformly convex regularizer and a \(q\)-th order oracle to solve the problem. This algorithm is applicable for all \(p, q\geq1\) and does not require a bisection search at each iteration.
3. **Complexity Analysis**: In addition, the paper also provides almost - matching lower bounds, which are applicable to any deterministic algorithm that interacts with a local oracle. The difference between the upper and lower bounds is only in the constants and logarithmic factors.
### Key Contributions
- **Generality**: The proposed algorithm solves the cases for all \(p, q\geq1\), not just specific values of \(p\) or \(q\).
- **Efficiency Improvement**: By using an inexact uniformly convex regularizer, the problem of requiring a bisection search at each iteration is avoided, improving the efficiency of the algorithm.
- **Theoretical Guarantee**: Almost - optimal complexity upper bounds are provided, and the corresponding lower bounds are proved by constructing hard instances, ensuring the effectiveness and optimality of the algorithm.
### Application Background
These problems have wide applications in fields such as deep learning, biochemistry and medicine. For example, in deep learning, when optimizing complex loss functions, the information of high - order derivatives can help adjust model parameters more accurately; in the field of biochemistry and medicine, when optimizing drug molecular structures or predicting biological activities, similar high - order smooth convex optimization problems also need to be dealt with.
In conclusion, through introducing new algorithms and techniques, this paper solves the high - order smooth convex optimization problem in non - Euclidean space, providing an important theoretical basis and practical tool for research in related fields.