Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization

Feihu Huang

2024-07-25

Abstract:Bilevel optimization is widely applied in many machine learning tasks such as hyper-parameter learning, meta learning and reinforcement learning. Although many algorithms recently have been developed to solve the bilevel optimization problems, they generally rely on the (strongly) convex lower-level problems. More recently, some methods have been proposed to solve the nonconvex-PL bilevel optimization problems, where their upper-level problems are possibly nonconvex, and their lower-level problems are also possibly nonconvex while satisfying Polyak-Łojasiewicz (PL) condition. However, these methods still have a high convergence complexity or a high computation complexity such as requiring compute expensive Hessian/Jacobian matrices and its inverses. In the paper, thus, we propose an efficient Hessian/Jacobian-free method (i.e., HJFBiO) with the optimal convergence complexity to solve the nonconvex-PL bilevel problems. Theoretically, under some mild conditions, we prove that our HJFBiO method obtains an optimal convergence rate of $O(\frac{1}{T})$, where $T$ denotes the number of iterations, and has an optimal gradient complexity of $O(\epsilon^{-1})$ in finding an $\epsilon$-stationary solution. We conduct some numerical experiments on the bilevel PL game and hyper-representation learning task to demonstrate efficiency of our proposed method.

Optimization and Control,Machine Learning

What problem does this paper attempt to address?

The paper focuses on addressing the challenges associated with nonconvex bilevel optimization problems, particularly those with nonconvex lower-level problems that satisfy the Polyak-Łojasiewicz (PL) condition. Bilevel optimization is a type of optimization problem where one optimization problem (the lower-level problem) is embedded within another (the upper-level problem). These problems are prevalent in various machine learning applications, including hyperparameter learning, meta-learning, and reinforcement learning.

### Problem Statement

The authors consider a class of nonconvex bilevel optimization problems of the form:

\[
\min_{x \in \mathbb{R}^d, y \in y^*(x)} f(x, y) + \phi(x)
\]
subject to
\[
y^*(x) \equiv \arg \min_{y \in \mathbb{R}^p} g(x, y),
\]

where:
- \(f(x, y)\) is the upper-level function, which can be nonconvex.
- \(\phi(x)\) is a convex but possibly nonsmooth regularization term.
- \(g(x, y)\) is the lower-level function, which can also be nonconvex but satisfies the PL condition.
- \(y^*(x)\) is the solution to the lower-level problem for a given \(x\).

### Challenges and Contributions

The paper addresses the inherent challenges of bilevel optimization, such as the intertwined nature of the upper and lower-level objectives via the minimizer \(y^*(x)\). Existing methods often rely on strong convexity assumptions for the lower-level problem, which the authors aim to relax by considering the nonconvex-PL setting.

The main contributions of the paper inc