T. Giovannelli,G. D. Kent,L. N. Vicente,T. GiovannelliG. D. KentL. N. VicenteDepartment of Industrial and Systems Engineering,Lehigh University,Bethlehem,PA,USAT. Giovannelli is a postdoctoral researcher and an adjunct faculty member in the Department of Industrial and Systems Engineering (ISE) at Lehigh University. He received his Ph.D. in Operations Research in May 2021 from Sapienza University of Rome,where he obtained his B.S. and M.S. in Engineering Management in 2015 and 2017,respectively. From January 2020 to May 2021,he was a visiting scholar at Lehigh ISE. In July 2019,he won a doctoral award at the Doctoral Consortium ORAHS 2019 (EURO Working Group within IFORS) in Karlsruhe,Germany,for his research in simulation-based optimization applied to emergency department management. His research interests span several sub-fields of nonlinear optimization,including stochastic bilevel and multiobjective optimization and simulation-based and derivative-free optimization. His main application areas include machine learning,transportation systems,and healthcare management.G. D. Kent is a Ph.D. candidate in the Department of Industrial and Systems Engineering at Lehigh University. He received his M.S. in Management Science and Engineering from Lehigh University and his B.S. in Statistics with a Minor in Computer Science from Utah Valley University. His research interests include stochastic nonlinear optimization,multi-level and multi-objective optimization,and machine/deep learning.L. N. Vicente is the Timothy J. Wilmott '80 Endowed Faculty Professor and Chair of Lehigh University's Department of Industrial and Systems Engineering (ISE),effective August 1,2018,after a career as a faculty member in the Department of Mathematics of the University of Coimbra. He obtained his PhD from Rice University in 1996,under a Fulbright scholarship,receiving from Rice the Ralph Budd Thesis Award. He was one of the three finalists of the 94-96 A. W. Tucker Prize of the Mathematical Optimization Society (MOS). In 2015,he was awarded the Lagrange Prize of SIAM (Society for Industrial and Applied Mathematics) and MOS for the co-authorship of the book "Introduction to Derivative-Free Optimization,MPS-SIAM Series on Optimization,SIAM,Philadelphia,2009". His research interests include Continuous Optimization,Computational Science and Engineering,and Machine Learning and Data Science. He was elected chair of the SIAM Activity Group on Optimization for 2023–2025.

Abstract:In this work, we propose different formulations and gradient-based algorithms for deterministic and stochastic bilevel problems with conflicting objectives in the lower level. Such problems have received little attention in the deterministic case and have never been studied from a stochastic approximation viewpoint despite the recent advances in stochastic methods for single-level, bilevel, and multi-objective optimization. To solve bilevel problems with a multi-objective lower level, different approaches can be considered depending on the interpretation of the lower-level optimality. An optimistic formulation that was previously introduced for the deterministic case consists of minimizing the upper-level function over all non-dominated lower-level solutions. In this paper, we develop new risk-neutral and risk-averse formulations, address their main computational challenges, and develop the corresponding deterministic and stochastic gradient-based algorithms.

Riemannian Bilevel Optimization

Riemannian Bilevel Optimization

A Framework for Bilevel Optimization on Riemannian Manifolds

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms

First-Order Methods for Linearly Constrained Bilevel Optimization

Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds

A Riemannian optimization method on the indefinite Stiefel manifold

Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms

Learning to Optimize on Riemannian Manifolds

Double Momentum Method for Lower-Level Constrained Bilevel Optimization

LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace

Efficient gradient-based methods for bilevel learning via recycling Krylov subspaces

Strong variational sufficiency of nonsmooth optimization problems on Riemannian manifolds

Riemannian Stochastic Hybrid Gradient Algorithm for Nonconvex Optimization

Riemannian optimization on the symplectic Stiefel manifold using second-order information

Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions

Bilevel optimization with a multi-objective lower-level problem: risk-neutral and risk-averse formulations