A note on optimality conditions for bilevel programming problems

Jane J. Ye,Daoli Zhu

DOI: https://doi.org/10.1080/02331939708844290

IF: 2.2

1997-01-01

Optimization

Abstract:In this note we correct a mistake In our paper “Optimality conditions for blevel programming problems” by giving two correct replacements

What problem does this paper attempt to address?

A note on the paper "Sufficient optimality conditions using convexifactors for optimistic bilevel programming problem"

Nazih Abderrazzak Gadhi

DOI: https://doi.org/10.3934/jimo.2021103

2021-01-01

Journal of Industrial and Management Optimization

Abstract:<p>In this work, some reasoning's mistakes in the paper by Kohli (doi:<a href="http://dx.doi.org/10.3934/jimo.2020114" target="_blank">10.3934/jimo.2020114</a>) are highlighted. Furthermore, we correct the flaws, propose a correct formulation of the main result (Theorem 5.1) and give alternative proofs.</p>

engineering, multidisciplinary,operations research & management science,mathematics, interdisciplinary applications
Optimality Conditions for Bilevel Programming Problems

J. J. Ye,D. L. Zhu

DOI: https://doi.org/10.1080/02331939508844060

IF: 2.2

1995-01-01

Optimization

Abstract:The bilevel programming problem (BLPP) is a sequence of two optimization problems where the constraint region of the upper level problem is determined implicitly by the solution set to the lower level problem. To obtain optimality conditions, we reformulate BLPP as a single level mathematical programming problem (SLPP) which involves the value function of the lower level problem. For this mathematical programming problem, it is shown that in general the usual constraint qualifications do not hold and the right constraint qualification is the calmness condition. It is also shown that the linear bilevel programming problem and the minmax problem satisfy the calmness condition automatically. A sufficient condition for the calmness for the bilevel programming problem with quadratic lower level problem and nondegenerate linear complementar¬ity lower level problem are given. First order necessary optimality condition are given using nonsmooth analysis. Second order sufficient optimality conditions are also given for the case where the lower level problem is unconstrained.
Optimality conditions for bilevel programmes via Moreau envelope reformulation*

Kuang Bai,Jane J. Ye,Shangzhi Zeng

DOI: https://doi.org/10.1080/02331934.2024.2358086

IF: 2.2

2024-05-29

Optimization

Abstract:For bilevel programmes with a convex lower-level programme, the classical approach replaces the lower-level programme with its Karush-Kuhn-Tucker condition and solve the resulting mathematical programme with complementarity constraint (MPCC). It is known that when the set of lower-level multipliers is not unique, MPCC may not be equivalent to the original bilevel problem, and many MPCC-tailored constraint qualifications do not hold. In this paper, we study bilevel programmes where the lower level is generalized convex. Applying the equivalent reformulation via Moreau envelope, we derive new directional optimality conditions. Even in the nondirectional case, the new optimality condition is stronger than the strong stationarity for the corresponding MPCC.

mathematics, applied,operations research & management science
New Necessary Optimality Conditions for Bilevel Programs by Combining the MPEC and Value Function Approaches

Jane J. Ye,Daoli Zhu

DOI: https://doi.org/10.1137/080725088

IF: 2.763

2013-01-01

SIAM Journal on Optimization

Abstract:The bilevel program is a sequence of two optimization problems where the constraint region of the upper level problem is determined implicitly by the solution set to the lower level problem. The classical approach to solving such a problem is to replace the lower level problem by its Karush-Kuhn-Tucker (KKT) condition and solve the resulting mathematical programming problem with equilibrium constraints (MPEC). In general the classical approach is not valid for nonconvex bilevel programming problems. The value function approach uses the value function of the lower level problem to define an equivalent single level problem. But the resulting problem requires a strong assumption, such as the partial calmness condition, for the KKT condition to hold. In this paper we combine the classical and the value function approaches to derive new necessary optimality conditions under rather weak conditions. The required conditions are even weaker in the case where the classical approach or the value function approach alone is applicable.
First order necessary optimality conditions for a class of pessimistic bilevel programming problems

Bing-bing LIU,Qing-yi HAO

DOI: https://doi.org/10.6040/j.issn.1671-9352.0.2015.091

2016-01-01

Abstract:In this paper,we investigate a class of pessimistic bilevel programming problem,where the upper-level prob-lem consists of a finite number of inequalities constraints and the lower-level problem is a cone constrained optimization problem.Firstly,using the maximization optimal value function of the upper-level problem and the minimization opti-mal value function of the lower-level problem,we translate the original problem into a single-level constrained optimiza-tion problem.Under some suitable assumptions,considering the subdifferential estimate of the maximization optimal value function and the properties of the upper and lower bound of the minimization optimal value function’s directional derivative,we obtain the detailed first-order necessary optimality conditions for the original problem.
Notes on "linear Programming Technique to Solve Two-Person Matrix Games with Interval Pay-Offs".

Deng-Feng Li

DOI: https://doi.org/10.1142/s021759591100351x

2011-01-01

Asia-Pacific Journal of Operational Research

Abstract:The aim of this note is to point out and correct some vital mistakes in the paper by P K Nayak and M Pal, "Linear programming technique to solve two person matrix (games with interval pay-offs). Asia-Pacific Journal of Operational Research, 26(2), 285–305". Lots of serious mistakes on the definitions, conclusions, models, methods, proofs and computing results have been corrected and modified in this note. We also indicate inappropriate formulations regarding their proposed linear programming models for solving generic matrix games with interval pay-offs and suggest a pair of linear programming models with any minimal acceptance degree of the interval inequality constraints which may be allowed to violate. The lexicographic method is suggested so that a rational and credible solution of the generic matrix game with interval pay-offs can be achieved.
An Enhanced Branch-and-bound Algorithm for Bilevel Integer Linear Programming.

Shaonan Liu,Mingzheng Wang,Nan Kong,Xiangpei Hu

DOI: https://doi.org/10.1016/j.ejor.2020.10.002

IF: 6.4

2020-01-01

European Journal of Operational Research

Abstract:Bilevel integer linear programming (BILP) problems have been studied for decades. Many exact algorithms have been proposed in recent years for small- or medium-sized instances. However, few of these algorithms were shown to be efficient on large-sized instances. In this paper, we present an enhanced branch-and-bound algorithm for a class of BILP problems, which can discard a subspace from the search space in each iteration larger than that in a benchmark branch-and-bound algorithm. The corresponding enhanced branching rule can efficiently slow down the creation of new node problems so as to significantly reduce the computation time. Our scheme may be suboptimal if the lower-level problem is not unique optimal as the enhanced branching rule may discard bilevel feasible solutions that may turn out to be optimal to the bilevel programming. We present computational studies to evaluate the algorithm speedup and solution quality of our algorithm, compared with state-of-the-art algorithms from the literature on a large testbed of general BILP instances, some of which are still unsolved. The computational results show that our enhanced branching rule can achieve significant speedup on the benchmark branching rule with satisfying solution quality. In particular, our algorithm shows superior performance on large-sized BILP instances with a relatively complex lower-level problem. (C) 2020 Elsevier B.V. All rights reserved.
PLMEs and Disjunctive Decompositions for Bilevel Optimization

Jiawang Nie,Jane J. Ye,Suhan Zhong

2023-04-03

Abstract:This paper studies bilevel polynomial optimization in which lower level constraining functions depend linearly on lower level variables. We show that such a bilevel program can be reformulated as a disjunctive program using partial Lagrange multiplier expressions (PLMEs). An advantage of this approach is that branch problems of the disjunctive program are easier to solve. In particular, since the PLME can be easily obtained, these branch problems can be efficiently solved by polynomial optimization techniques. Solving each branch problem either returns infeasibility or gives a candidate local or global optimizer for the original bilevel optimization. We give necessary and sufficient conditions for these candidates to be global optimizers, and sufficient conditions for the local optimality. Numerical experiments are also presented to show the efficiency of the method.

Optimization and Control
Correction to: A New Abadie-Type Constraint Qualification for General Optimization Problems

M. Alavi Hejazi,N. Movahedian

DOI: https://doi.org/10.1007/s10957-023-02298-x

2023-10-22

Journal of Optimization Theory and Applications

Abstract:The publication " J. Optim. Theory Appl. 186, 86–101 (2020). https://doi.org/10.1007/s10957-020-01691-0 " requires minor modifications which are carried out.

mathematics, applied,operations research & management science
Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs

Lucy L. Gao,Jane J. Ye,Haian Yin,Shangzhi Zeng,Jin Zhang

2024-01-21

Abstract:Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (2023), a value function-based difference of convex algorithm was introduced to address bilevel programs. This approach proves particularly powerful when dealing with scenarios where the lower-level problem exhibits convexity in both the upper-level and lower-level variables. Examples of such scenarios include support vector machines and $\ell_1$ and $\ell_2$ regularized regression. In this paper, we significantly expand the range of applications, now requiring convexity only in the lower-level variables of the lower-level program. We present an innovative single-level difference of weakly convex reformulation based on the Moreau envelope of the lower-level problem. We further develop a sequentially convergent Inexact Proximal Difference of Weakly Convex Algorithm (iP-DwCA). To evaluate the effectiveness of the proposed iP-DwCA, we conduct numerical experiments focused on tuning hyperparameters for kernel support vector machines on simulated data.

Optimization and Control,Machine Learning
A partial cooperation model for bilevel programming problems with finite reactions follower

LIU Bingbing,WAN Zhongping

DOI: https://doi.org/10.3969/j.issn.1007-6093.2012.03.005

2012-01-01

Abstract:Bilevel programming problem provide a framework to deal with decision processes involving two decision makers with a hierarchical structure.The leader at the upper level of the hierarchy and the follower at the lower level seek to optimize their individual objective functions and control their own set of decision variables.Bilevel programming problem involves two optimization problems where the constraint region of the upper level problem is implicitly determined by another optimization problem.Bilevel programming problem is frequently encountered with in the fields of economy,industry, transportation,military,and so on.In this paper,we investigate the partial cooperation model for the bilevel programming problem in which the optimal reactions of the lower level problem is discrete and finite.We develop a general method for ascertaining the cooperation ratio in the partial cooperation model when the cooperation degree of the follower depends on the decision variables of the leader.Furthermore,we develop a new partial cooperation model of which the optimal is better than of the pessimistic model under suitable conditions.Then we derive some meaningful results based the proposed partial cooperation model.Finally,we show that the proposed partial cooperation model is feasible by two numerical examples.
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis

Jie Hao,Xiaochuan Gong,Mingrui Liu

2024-01-18

Abstract:Bilevel optimization is an important formulation for many machine learning problems. Current bilevel optimization algorithms assume that the gradient of the upper-level function is Lipschitz. However, recent studies reveal that certain neural networks such as recurrent neural networks (RNNs) and long-short-term memory networks (LSTMs) exhibit potential unbounded smoothness, rendering conventional bilevel optimization algorithms unsuitable. In this paper, we design a new bilevel optimization algorithm, namely BO-REP, to address this challenge. This algorithm updates the upper-level variable using normalized momentum and incorporates two novel techniques for updating the lower-level variable: \textit{initialization refinement} and \textit{periodic updates}. Specifically, once the upper-level variable is initialized, a subroutine is invoked to obtain a refined estimate of the corresponding optimal lower-level variable, and the lower-level variable is updated only after every specific period instead of each iteration. When the upper-level problem is nonconvex and unbounded smooth, and the lower-level problem is strongly convex, we prove that our algorithm requires $\widetilde{\mathcal{O}}(1/\epsilon^4)$ iterations to find an $\epsilon$-stationary point in the stochastic setting, where each iteration involves calling a stochastic gradient or Hessian-vector product oracle. Notably, this result matches the state-of-the-art complexity results under the bounded smoothness setting and without mean-squared smoothness of the stochastic gradient, up to logarithmic factors. Our proof relies on novel technical lemmas for the periodically updated lower-level variable, which are of independent interest. Our experiments on hyper-representation learning, hyperparameter optimization, and data hyper-cleaning for text classification tasks demonstrate the effectiveness of our proposed algorithm.

Machine Learning,Optimization and Control
A GENETIC ALGORITHM BASED ON OPTIMALITY CONDITIONS FOR NONLINEAR BILEVEL PROGRAMMING PROBLEMS

Hecheng Li,Yuping Wang

2010-01-01

Journal of applied mathematics & informatics

Abstract:For a class of nonlinear bilevel programming problems in which the follower's problem is linear, the paper develops a genetic algorithm based on the optimality conditions of linear programming. At first, we denote an individual by selecting a base of the follower's linear programming, and use the optimality conditions given in the simplex method to denote the follower's solution functions. Then, the follower's problem and variables are replaced by these optimality conditions and the solution functions, which makes the original bilevel programming become a single-level one only including the leader's variables. At last, the single-level problem is solved by using some classical optimization techniques, and its objective value is regarded as the fitness of the individual. The numerical results illustrate that the proposed algorithm is efficient and stable.
Solving Bilevel Programs Based on Lower-Level Mond-Weir Duality

Yu-Wei Li,Gui-Hua Lin,Xide Zhu

DOI: https://doi.org/10.1287/ijoc.2023.0108

IF: 3.288

2024-02-16

INFORMS Journal on Computing

Abstract:This paper focuses on developing effective algorithms for solving a bilevel program. The most popular approach is to replace the lower-level problem with its Karush-Kuhn-Tucker conditions to generate a mathematical program with complementarity constraints (MPCC). However, MPCC does not satisfy the Mangasarian-Fromovitz constraint qualification (MFCQ) at any feasible point. In this paper, inspired by a recent work using the lower-level Wolfe duality (WDP), we apply the lower-level Mond-Weir duality to present a new reformulation, called MDP, for bilevel program. It is shown that, under mild assumptions, they are equivalent in globally or locally optimal sense. An example is given to show that, different from MPCC, MDP may satisfy the MFCQ at its feasible points. Relations among MDP, WDP, and MPCC are investigated. On this basis, we extend the MDP reformulation to present another new reformulation (called eMDP), which has similar properties to MDP. Furthermore, to compare two new reformulations with the MPCC and WDP approaches, we design a procedure to generate 150 tested problems randomly and comprehensive numerical experiments show that MDP has quite evident advantages over MPCC and WDP in terms of feasibility to the original bilevel programs, success efficiency, and average CPU time, whereas eMDP is far superior to all other three reformulations. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work was supported by the National Natural Science Foundation of China [Grants 12071280 and 11901380]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0108 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0108 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

computer science, interdisciplinary applications,operations research & management science
Robust Bilevel Optimization for Near-Optimal Lower-Level Solutions

Mathieu Besançon,Miguel F. Anjos,Luce Brotcorne

2024-07-12

Abstract:Bilevel optimization problems embed the optimality of a subproblem as a constraint of another optimization problem. We introduce the concept of near-optimality robustness for bilevel optimization, protecting the upper-level solution feasibility from limited deviations from the optimal solution at the lower level. General properties and necessary conditions for the existence of solutions are derived for near-optimal robust versions of general bilevel optimization problems. A duality-based solution method is defined when the lower level is convex, leveraging the methodology from the robust and bilevel literature. Numerical results assess the efficiency of exact and heuristic methods and the impact of valid inequalities on the solution time.

Optimization and Control,Computer Science and Game Theory
Correction to: A Lagrangian dual method for two-stage robust optimization with binary uncertainties

Henri Lefebvre,Anirudh Subramanyam

2024-11-07

Abstract:We provide a correction to the sufficient conditions under which closed-form expressions for the optimal Lagrange multiplier are provided in <a class="link-https" data-arxiv-id="2112.13138" href="https://arxiv.org/abs/2112.13138">arXiv:2112.13138</a> [math.OC]. We first present a simple counterexample where the original conditions are insufficient, highlight where the original proof fails, and then provide modified conditions along with a correct proof of their validity. Finally, although the original paper discusses modifications to their method for problems that may not satisfy any sufficient conditions, we substantiate that discussion along two directions. We first show that computing an optimal Lagrange multiplier can still be done in polynomial time. We then provide complete and correct versions of the corresponding Benders and column-and-constraint generation algorithms in which the original method is used. We also discuss the implications of our findings on computational performance.

Optimization and Control
Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems

Huaqing Zhang,Lesi Chen,Jing Xu,Jingzhao Zhang

2024-09-10

Abstract:This paper studies simple bilevel problems, where a convex upper-level function is minimized over the optimal solutions of a convex lower-level problem. We first show the fundamental difficulty of simple bilevel problems, that the approximate optimal value of such problems is not obtainable by first-order zero-respecting algorithms. Then we follow recent works to pursue the weak approximate solutions. For this goal, we propose novel near-optimal methods for smooth and nonsmooth problems by reformulating them into functionally constrained problems.

Optimization and Control,Machine Learning
ε-exact Penalty for Non-concave Bilevel Programming Problems

Li Changmin,Zhu Daoli

2011-01-01

Acta Mathematica Scientia

Abstract:The main tool in the present literature to treat the bilevel programming problems (BLPP) is the method of KKT conditions and value function.However,for the non-concave case,neither technique can be applied.In this paper,the authors introduce the notions ofε-set, e-error bounds,and use certainε-uniform error bounds asε-exact penalties to give single level problems equivalent to the approximate BLPP.Furthermore,they show that any cluster of the approximate sequence is the solution of the BLPP.
A single-level reformulation of mixed integer bilevel programming problems.

Chuanjia Li,Lei Guo

DOI: https://doi.org/10.1016/j.orl.2016.10.013

2017-01-01

Abstract:This paper focuses on the single-level reformulation of mixed integer bilevel programming problems (MIBLPP). Due to the existence of lower-level integer variables, the popular approaches in the literature such as the first-order approach are not applicable to the MIBLPP. In this paper, we reformulate the MIBLPP as a mixed integer mathematical program with complementarity constraints (MIMPCC) by separating the lower-level continuous and integer variables. In particular, we show that global and local minimizers of the MIBLPP correspond to those of the MIMPCC respectively under suitable conditions.
A Homotopy Method for Solving Bilevel Programming Problem

DL Zhu,Q Xu,ZH Lin

DOI: https://doi.org/10.1016/j.na.2004.03.022

IF: 1.743

2004-01-01

Nonlinear Analysis

Abstract:A homotopy method for solving bilevel programming problem (BLPP) is proposed. The algorithm is based on constructing the homotopy equation for BLPP. One satisfiable sufficient condition is given to ensure that the point of homotopy path globally converges to a K–K–T point of BLPP. Numerical results are presented to show the effectiveness of this method.

A note on optimality conditions for bilevel programming problems

A note on the paper "Sufficient optimality conditions using convexifactors for optimistic bilevel programming problem"

Optimality Conditions for Bilevel Programming Problems

Optimality conditions for bilevel programmes via Moreau envelope reformulation*

New Necessary Optimality Conditions for Bilevel Programs by Combining the MPEC and Value Function Approaches

First order necessary optimality conditions for a class of pessimistic bilevel programming problems

Notes on "linear Programming Technique to Solve Two-Person Matrix Games with Interval Pay-Offs".

An Enhanced Branch-and-bound Algorithm for Bilevel Integer Linear Programming.

PLMEs and Disjunctive Decompositions for Bilevel Optimization

Correction to: A New Abadie-Type Constraint Qualification for General Optimization Problems

Moreau Envelope Based Difference-of-weakly-Convex Reformulation and Algorithm for Bilevel Programs

A partial cooperation model for bilevel programming problems with finite reactions follower

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis

A GENETIC ALGORITHM BASED ON OPTIMALITY CONDITIONS FOR NONLINEAR BILEVEL PROGRAMMING PROBLEMS

Solving Bilevel Programs Based on Lower-Level Mond-Weir Duality

Robust Bilevel Optimization for Near-Optimal Lower-Level Solutions

Correction to: A Lagrangian dual method for two-stage robust optimization with binary uncertainties

Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems

ε-exact Penalty for Non-concave Bilevel Programming Problems

A single-level reformulation of mixed integer bilevel programming problems.

A Homotopy Method for Solving Bilevel Programming Problem