Abstract:Bilevel optimization provides a comprehensive framework that bridges single- and multi-objective optimization, encompassing various formulations, including standard nonlinear programs. This paper focuses on a specific class of bilevel optimization known as simple bilevel optimization. In these problems, the objective is to minimize a composite convex function over the optimal solution set of another composite convex minimization problem. By reformulating the simple bilevel problem as finding the left-most root of a nonlinear equation, we employ a bisection scheme to efficiently obtain a solution that is $\epsilon$-optimal for both the upper- and lower-level objectives. In each iteration, the bisection narrows down an interval by assessing the feasibility of a discriminating criterion. By introducing a novel dual approach and employing the Accelerated Proximal Gradient (APG) method, we demonstrate that each subproblem in the bisection scheme can be solved in ${\mathcal{O}}(\sqrt{(L_{g_1}+2D_z L_{f_1}+1)/\epsilon}|\log\epsilon|^2)$ oracle queries under weak assumptions. Here, $L_{f_1}$ and $L_{g_1}$ represent the Lipschitz constants of the gradients of the upper- and lower-level objectives' smooth components, and $D_z$ is the upper bound of the optimal multiplier of the subproblem. Considering the number of binary searches, the total complexity of our proposed method is ${\mathcal{O}}(\sqrt{(L_{g_1}+2D_z L_{f_1}+1)/\epsilon}|\log\epsilon|^3)$. Our method achieves near-optimal complexity results, comparable to those in unconstrained smooth or composite convex optimization when disregarding the logarithmic terms. Numerical experiments also demonstrate the superior performance of our method compared to the state-of-the-art.

Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems

A Near-Optimal Algorithm for Convex Simple Bilevel Optimization under Weak Assumptions

Near-Optimal Convex Simple Bilevel Optimization with a Bisection Method

Barrier Function for Bilevel Optimization with Coupled Lower-Level Constraints: Formulation, Approximation and Algorithms

A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem

An Approximate Globally Convergent Method for Solving Linear Bilevel Programms

An Alternating Linearization Method with Inexact Data for Bilevel Nonsmooth Convex Optimization

An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization

Penalty-based Methods for Simple Bilevel Optimization under Hölderian Error Bounds

An Algorithm for Solving Bilevel Linear Programming Problem

A Value-Function-Based Exact Approach for the Bilevel Mixed-Integer Programming Problem

An Inexact Conditional Gradient Method for Constrained Bilevel Optimization

An Algorithm for the Lower Bounds of Semivectorial Bilevel Programming Problem

Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem

An Enhanced Branch-and-bound Algorithm for Bilevel Integer Linear Programming.

Bilevel Optimization without Lower-Level Strong Convexity from the Hyper-Objective Perspective

Robust Bilevel Optimization for Near-Optimal Lower-Level Solutions

Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm

A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints

Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles

Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions