Universal Online Convex Optimization Meets Second-order Bounds

Lijun Zhang,Yibo Wang,Guanghui Wang,Jinfeng Yi,Tianbao Yang

2024-11-20

Abstract:Recently, several universal methods have been proposed for online convex optimization, and attain minimax rates for multiple types of convex functions simultaneously. However, they need to design and optimize one surrogate loss for each type of functions, making it difficult to exploit the structure of the problem and utilize existing algorithms. In this paper, we propose a simple strategy for universal online convex optimization, which avoids these limitations. The key idea is to construct a set of experts to process the original online functions, and deploy a meta-algorithm over the linearized losses to aggregate predictions from experts. Specifically, the meta-algorithm is required to yield a second-order bound with excess losses, so that it can leverage strong convexity and exponential concavity to control the meta-regret. In this way, our strategy inherits the theoretical guarantee of any expert designed for strongly convex functions and exponentially concave functions, up to a double logarithmic factor. As a result, we can plug in off-the-shelf online solvers as black-box experts to deliver problem-dependent regret bounds. For general convex functions, it maintains the minimax optimality and also achieves a small-loss bound. Furthermore, we extend our universal strategy to online composite optimization, where the loss function comprises a time-varying function and a fixed regularizer. To deal with the composite loss functions, we employ a meta-algorithm based on the optimistic online learning framework, which not only possesses a second-order bound, but also can utilize estimations for upcoming loss functions. With appropriate configurations, we demonstrate that the additional regularizer does not contribute to the meta-regret, thus maintaining the universality in the composite setting.

Machine Learning,Optimization and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to develop a general and simple strategy in Online Convex Optimization (OCO) to overcome the limitations of existing methods. Specifically, although existing general methods can handle multiple types of convex functions (such as general convex functions, strongly convex functions, and exponentially concave functions), they need to design and optimize a surrogate loss for each type of function, which makes it difficult to utilize the structure of the problem and use existing algorithms. Moreover, methods relying on surrogate losses are difficult to generate regret bounds related to the problem, except for the small - loss bound.

For this reason, the paper proposes a new strategy. It constructs a set of experts to handle the original online functions and uses a meta - algorithm to aggregate predictions on the linearized losses of these experts. The key points are:

1. **Experts handle the original functions**: Each expert directly handles the original online function and can utilize the structure of the problem instance to generate regret bounds related to the problem.
2. **Meta - algorithm uses linearized losses**: The meta - algorithm selects linearized losses to measure the performance of experts and uses second - order bounds to control meta - regret.

The advantages of this strategy are:
- **Decoupling the losses of experts and meta - algorithms**: The losses used by experts and meta - algorithms are separated, and existing online algorithms can be directly used as black - box programs without the need to design surrogate losses.
- **Applicable to multiple types of functions**: For strongly convex functions and exponentially concave functions, this strategy can achieve the optimal regret bounds, provided that the domain and gradient are bounded.
- **Extension to composite online optimization**: This strategy can also be extended to Online Composite Optimization, where the loss function consists of a time - varying function and a fixed regularization term.

In this way, the method proposed in the paper not only inherits the theoretical guarantees of existing expert algorithms but also can achieve problem - related regret bounds on different types of functions, thereby improving the generality and flexibility of online convex optimization.