A decomposition algorithm for two-stage stochastic programs with approximate rotational invariance

Marzieh Bakhshi,Konstantin Tikhomirov

2024-07-31

Abstract:We propose an algorithm of approximating the optimal objective value of a two-stage stochastic program under an assumption of {\it approximate rotational invariance} of the technology matrix, and compare the method with the L-shaped decomposition.

Optimization and Control

What problem does this paper attempt to address?

Online Successive Convex Approximation for Two-Stage Stochastic Nonconvex Optimization

An Liu,Vincent K. N. Lau,Min-Jian Zhao

DOI: https://doi.org/10.1109/tsp.2018.2871389

IF: 4.875

2018-01-01

IEEE Transactions on Signal Processing

Abstract:Two-stage stochastic optimization, in which a long-term master problem is coupled with a family of short-term subproblems, plays a critical role in various application areas. However, most existing algorithms for two-stage stochastic optimization only work for special cases, and/or are based on the batch method, which requires huge memory and computational complexity. To the best of our knowledge, there still lack efficient and general two-stage online stochastic optimization algorithms. This paper proposes a two-stage online successive convex approximation (TOSCA) algorithm for general two-stage nonconvex stochastic optimization problems. At each iteration, the TOSCA algorithm first solves one short-term subproblem associated with the current realization of the system state. Then, it constructs a convex surrogate function for the objective of the long-term master problem. Finally, the long-term variables are updated by solving a convex approximation problem obtained by replacing the objective function in the long-term master problem with the convex surrogate function. We establish the almost sure convergence of the TOSCA algorithm and customize the algorithmic framework to solve three important application problems. Simulations show that the TOSCA algorithm can achieve superior performance over existing solutions.
A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse Functions

Hanyang Li,Ying Cui

DOI: https://doi.org/10.1137/22m1488533

IF: 2.763

2024-01-20

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 1, Page 306-335, March 2024. In this paper, we have studied a decomposition method for solving a class of nonconvex two-stage stochastic programs, where both the objective and constraints of the second-stage problem are nonlinearly parameterized by the first-stage variables. Due to the failure of the Clarke regularity of the resulting nonconvex recourse function, classical decomposition approaches such as Benders decomposition and (augmented) Lagrangian-based algorithms cannot be directly generalized to solve such models. By exploring an implicitly convex-concave structure of the recourse function, we introduce a novel decomposition framework based on the so-called partial Moreau envelope. The algorithm successively generates strongly convex quadratic approximations of the recourse function based on the solutions of the second-stage convex subproblems and adds them to the first-stage master problem. Convergence has been established for both a fixed number of scenarios and a sequential internal sampling strategy. Numerical experiments are conducted to demonstrate the effectiveness of the proposed algorithm.

mathematics, applied
Approximate Truthful Mechanism Design For Two-Dimensional Orthogonal Knapsack Problem

Deshi Ye,Guochuan Zhang

DOI: https://doi.org/10.1007/978-3-319-21398-9_31

2015-01-01

Abstract:This paper provides a technique for designing truthful mechanisms for a combinatorial optimization problem, which requires composition algorithms. We show that the composition algorithm A o B is monotone if the algorithm A and the algorithm B are both monotone. Then, we apply this technique to the two-dimensional orthogonal knapsack problem with provable approximation bounds, improving the previous results in [5].
A Decomposition Method for Distributionally-Robust Two-stage Stochastic Mixed-integer Cone Programs

Fengqiao Luo,Sanjay Mehrotra

DOI: https://doi.org/10.48550/arXiv.1911.08713

2019-11-20

Optimization and Control

Abstract:We develop a decomposition algorithm for distributionally-robust two-stage stochastic mixed-integer convex cone programs, and its important special case of distributionally-robust two-stage stochastic mixed-integer second order cone programs. This generalizes the algorithm proposed by Sen and Sherali~[Mathematical Programming 106(2): 203-223, 2006]. We show that the proposed algorithm is finitely convergent if the second-stage problems are solved to optimality at incumbent first stage solutions, and solution to an optimization problem to identify worst-case probability distribution is available. The second stage problems can be solved using a branch-and-cut algorithm. The decomposition algorithm is illustrated with an example. Computational results on a stochastic programming generalization of a facility location problem show significant solution time improvements from the proposed approach. Solutions for many models that are intractable for an extensive form formulation become possible. Computational results suggest that solution time requirement does not increase significantly when considering distributional robust counterparts to the stochastic programming models.
Two-Stage Stochastic Optimization via Primal-Dual Decomposition and Deep Unrolling

An Liu,Rui Yang,Tony Q. S. Quek,Min-Jian Zhao

DOI: https://doi.org/10.1109/TSP.2021.3079807

2021-05-05

Abstract:We consider a two-stage stochastic optimization problem, in which a long-term optimization variable is coupled with a set of short-term optimization variables in both objective and constraint functions. Despite that two-stage stochastic optimization plays a critical role in various engineering and scientific applications, there still lack efficient algorithms, especially when the long-term and short-term variables are coupled in the constraints. To overcome the challenge caused by tightly coupled stochastic constraints, we first establish a two-stage primal-dual decomposition (PDD) method to decompose the two-stage problem into a long-term problem and a family of short-term subproblems. Then we propose a PDD-based stochastic successive convex approximation (PDD-SSCA) algorithmic framework to find KKT solutions for two-stage stochastic optimization problems. At each iteration, PDD-SSCA first runs a short-term sub-algorithm to find stationary points of the short-term subproblems associated with a mini-batch of the state samples. Then it constructs a convex surrogate for the long-term problem based on the deep unrolling of the short-term sub-algorithm and the back propagation method. Finally, the optimal solution of the convex surrogate problem is solved to generate the next iterate. We establish the almost sure convergence of PDD-SSCA and customize the algorithmic framework to solve two important application problems. Simulations show that PDD-SSCA can achieve superior performance over existing solutions.

Optimization and Control,Information Theory,Machine Learning
Asymptotic Results of Stochastic Decomposition for Two-Stage Stochastic Quadratic Programming

Junyi Liu,Suvrajeet Sen

DOI: https://doi.org/10.1137/19m1247796

IF: 2.763

2020-01-01

SIAM Journal on Optimization

Abstract:This paper presents the stochastic decomposition (SD) algorithms for two classes of stochastic programming problems: (1) two-stage stochastic quadratic-linear programming (SQLP) in which a quadratic program defines the objective function in the first stage and a linear program defines the value function in the second stage and (2) two-stage stochastic quadratic-quadratic programming (SQQP) which has quadratic programming problems in both stages. Similar to their stochastic linear programming (SLP) predecessor, these iterative schemes in SD approximate the objective function using piecewise affine/quadratic minorants and then apply a stochastic proximal mapping to obtain the next iterate. In this paper we show that under some assumptions, the proximal mapping applied in SD obeys a contraction mapping property even though the approximations are based on sequential random samples. Following that, we demonstrate that under those assumptions, SD can provide a sequence of solutions converging to the optimal solution with a sublinear convergence rate in both SQLP and SQQP problems. Finally, we present an “in-sample” stopping rule to assess the optimality gap by constructing consistent bootstrap estimators.
On the Convergence of L-shaped Algorithms for Two-Stage Stochastic Programming

John R. Birge,Haihao Lu,Baoyu Zhou

2023-09-04

Abstract:In this paper, we design, analyze, and implement a variant of the two-loop L-shaped algorithms for solving two-stage stochastic programming problems that arise from important application areas including revenue management and power systems. We consider the setting in which it is intractable to compute exact objective function and (sub)gradient information, and instead, only estimates of objective function and (sub)gradient values are available. Under common assumptions including fixed recourse and bounded (sub)gradients, the algorithm generates a sequence of iterates that converge to a neighborhood of optimality, where the radius of the convergence neighborhood depends on the level of the inexactness of objective function estimates. The number of outer and inner iterations needed to find an approximate optimal iterate is provided. Finally, we show a sample complexity result for the algorithm with a Polyak-type step-size policy that can be extended to analyze other situations. We also present a numerical study that verifies our theoretical results and demonstrates the superior empirical performance of our proposed algorithms over classic solvers.

Optimization and Control
Unified Branch-and-Benders-Cut for two-stage stochastic mixed-integer programs

Arthur Mahéo,Simon Belieres,Yossiri Adulyasak,Jean-François Cordeau

DOI: https://doi.org/10.1016/j.cor.2023.106526

IF: 5.159

2024-01-10

Computers & Operations Research

Abstract:Two-stage stochastic programs are a class of stochastic problems where uncertainty is discretized into scenarios, making them amenable to solution approaches such as Benders decomposition. However, classic Benders decomposition is not applicable to general two-stage stochastic mixed-integer programs due to the restriction that the second-stage variables should be continuous. We propose a novel Benders decomposition-based framework that accommodates mixed-integer variables in both stages as well as uncertainty in all of the recourse parameters. The proposed approach is a unified branch-and-Benders algorithm, where we use a heuristic to maintain a global upper bound and a post-processing phase to determine an optimal solution. This new approach is flexible, allowing practitioners to integrate acceleration techniques such as partial decomposition or convexification schemes. We demonstrate the efficiency of our approach versus classic ones on the stochastic server location problem, and its generality on a new, complex stochastic problem where the second stage is a traveling salesman problem.

computer science, interdisciplinary applications,engineering, industrial,operations research & management science
Towards Global Solutions for Nonconvex Two-Stage Stochastic Programs: A Polynomial Lower Approximation Approach

Suhan Zhong,Ying Cui,Jiawang Nie

DOI: https://doi.org/10.1137/23m1615516

IF: 2.763

2024-10-11

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 4, Page 3477-3505, December 2024. This paper tackles the challenging problem of finding global optimal solutions for two-stage stochastic programs with continuous decision variables and nonconvex recourse functions. We introduce a two-phase approach. The first phase involves the construction of a polynomial lower bound for the recourse function through a linear optimization problem over a nonnegative polynomial cone. Given the complex structure of this cone, we employ semidefinite relaxations with quadratic modules to facilitate our computations. In the second phase, we solve a surrogate first-stage problem by substituting the original recourse function with the polynomial lower approximation obtained in the first phase. Our method is particularly advantageous for two reasons: it not only generates global lower bounds for the nonconvex stochastic program, aiding in the certificate of global optimality for prospective solutions like stationary solutions computed from other methods, but it also yields an explicit polynomial approximation for the recourse function through the solution of a linear conic optimization problem, where the number of variables is independent of the support of the underlying random vector. Therefore, our approach is particularly suitable for the case where the random vector follows a continuous distribution or when dealing with a large number of scenarios. Numerical experiments are conducted to demonstrate the effectiveness of our proposed approach.

mathematics, applied
An Optimal Staging Algorithm

Takuma Murakami,Zhenjiang Hu,Masato Takeichi

2002-01-01

Abstract:Staging is an alternative approach for binding-time analysis and program specialization, making some pieces of code dynamic. Tim Sheard and Nathan Linger showed a search-based algorithm for auto- mated staging. The problems remained are unclear computation com- plexity and no ecient way to take optimal solutions. In this paper, we propose a linear algorithm to derive optimally staged programs with con- crete implementation. The key point of our approach is to make use of the optimization theorem for solving multi-marking problem by regard- ing staging as marking.
A decomposition algorithm for solving large-scale quadratic programming problems

Hui-Min Li,Ke-Cun Zhang

DOI: https://doi.org/10.1016/j.amc.2005.04.076

IF: 4.397

2006-02-01

Applied Mathematics and Computation

Abstract:In this paper an algorithm for solving large-scale programming is proposed. We decompose a large-scale quadratic programming into a serial of small-scale ones and then approximate the solution of the large-scale quadratic programming via the solutions of these small-scale ones. It is proved that the accumulation point of the iterates generated by the algorithm is a global minimum point of the quadratic programming. The algorithm has performed very well in numerical testing. It is a new way of solving large-scale quadratic programming problems.

mathematics, applied
Solving Two-Stage Stochastic Programs with Endogenous Uncertainty via Random Variable Transformation

Maria Bazotte,Margarida Carvalho,Thibaut Vidal

2024-07-18

Abstract:Real-world decision-making problems often involve decision-dependent uncertainty, where the probability distribution of the random vector depends on the model decisions. Few studies focus on two-stage stochastic programs with this type of endogenous uncertainty, and those that do lack general methodologies. We propose a general method for solving a class of these programs based on random variable transformation, a technique widely employed in probability and statistics. The random variable transformation converts a stochastic program with endogenous uncertainty (original program) into an equivalent stochastic program with decision-independent uncertainty (transformed program), for which solution procedures are well-studied. Additionally, endogenous uncertainty usually leads to nonlinear nonconvex programs, which are theoretically intractable. Nonetheless, we show that, for some classical endogenous distributions, the proposed method yields mixed-integer linear or convex programs with exogenous uncertainty. We validate this method by applying it to a network design and facility-protection problem, considering distinct decision-dependent distributions for the random variables. While the original formulation of this problem is nonlinear nonconvex for most endogenous distributions, the proposed method transforms it into mixed-integer linear programs with exogenous uncertainty. We solve these transformed programs with the sample average approximation method. We highlight the superior performance of our approach compared to solving the original program in the case a mixed-integer linear formulation of this program exists.

Optimization and Control
Efficient algorithms for Tucker decomposition via approximate matrix multiplication

Maolin Che,Yimin Wei,Hong Yan

2023-03-21

Abstract:This paper develops fast and efficient algorithms for computing Tucker decomposition with a given multilinear rank. By combining random projection and the power scheme, we propose two efficient randomized versions for the truncated high-order singular value decomposition (T-HOSVD) and the sequentially T-HOSVD (ST-HOSVD), which are two common algorithms for approximating Tucker decomposition. To reduce the complexities of these two algorithms, fast and efficient algorithms are designed by combining two algorithms and approximate matrix multiplication. The theoretical results are also achieved based on the bounds of singular values of standard Gaussian matrices and the theoretical results for approximate matrix multiplication. Finally, the efficiency of these algorithms are illustrated via some test tensors from synthetic and real datasets.

Numerical Analysis
An Efficient Randomized Algorithm for Computing the Approximate Tucker Decomposition

Che Maolin,Wei Yimin,Yan Hong

DOI: https://doi.org/10.1007/s10915-021-01545-5

2021-01-01

Journal of Scientific Computing

Abstract:By combining the thin QR decomposition and the subsampled randomized Fourier transform (SRFT), we obtain an efficient randomized algorithm for computing the approximate Tucker decomposition with a given target multilinear rank. We also combine this randomized algorithm with the power iteration technique to improve the efficiency of the algorithm. By using the results about the singular values of the product of orthonormal matrices with the Kronecker product of SRFT matrices, we obtain the error bounds of these two algorithms. Finally, the efficiency of these algorithms is illustrated by several numerical examples.
A randomized operator splitting scheme inspired by stochastic optimization methods

Monika Eisenmann,Tony Stillfjord

DOI: https://doi.org/10.1007/s00211-024-01396-w

IF: 2.5

2024-02-27

Numerische Mathematik

Abstract:In this paper, we combine the operator splitting methodology for abstract evolution equations with that of stochastic methods for large-scale optimization problems. The combination results in a randomized splitting scheme, which in a given time step does not necessarily use all the parts of the split operator. This is in contrast to deterministic splitting schemes which always use every part at least once, and often several times. As a result, the computational cost can be significantly decreased in comparison to such methods. We rigorously define a randomized operator splitting scheme in an abstract setting and provide an error analysis where we prove that the temporal convergence order of the scheme is at least 1/2. We illustrate the theory by numerical experiments on both linear and quasilinear diffusion problems, using a randomized domain decomposition approach. We conclude that choosing the randomization in certain ways may improve the order to 1. This is as accurate as applying e.g. backward (implicit) Euler to the full problem, without splitting.

mathematics, applied
Decomposition Algorithms for Stochastic Programming on a Computational Grid

Jeff Linderoth,Stephen Wright

DOI: https://doi.org/10.1023/a:1021858008222

2003-01-01

Computational Optimization and Applications

Abstract:We describe algorithms for two-stage stochastic linear programming with recourse and their implementation on a grid computing platform. In particular, we examine serial and asynchronous versions of the L-shaped method and a trust-region method. The parallel platform of choice is the dynamic, heterogeneous, opportunistic platform provided by the Condor system. The algorithms are of master-worker type (with the workers being used to solve second-stage problems), and the MW runtime support library (which supports master-worker computations) is key to the implementation. Computational results are presented on large sample-average approximations of problems from the literature.

mathematics, applied,operations research & management science
The decomposition-based outer approximation algorithm for convex mixed-integer nonlinear programming

Pavlo Muts,Ivo Nowak,Eligius M. T. Hendrix

DOI: https://doi.org/10.1007/s10898-020-00888-x

2020-02-20

Journal of Global Optimization

Abstract:Abstract This paper presents a new two-phase method for solving convex mixed-integer nonlinear programming (MINLP) problems, called Decomposition-based Outer Approximation Algorithm (DECOA). In the first phase, a sequence of linear integer relaxed sub-problems (LP phase) is solved in order to rapidly generate a good linear relaxation of the original MINLP problem. In the second phase, the algorithm solves a sequence of mixed integer linear programming sub-problems (MIP phase). In both phases the outer approximation is improved iteratively by adding new supporting hyperplanes by solving many easier sub-problems in parallel. DECOA is implemented as a part of Decogo (Decomposition-based Global Optimizer), a parallel decomposition-based MINLP solver implemented in Python and Pyomo. Preliminary numerical results based on 70 convex MINLP instances up to 2700 variables show that due to the generated cuts in the LP phase, on average only 2–3 MIP problems have to be solved in the MIP phase.

mathematics, applied,operations research & management science
Calculating the expected value function of a two-stage stochastic optimization program with a quantum algorithm

Caleb Rotello,Peter Graf,Matthew Reynolds,Cody James Winkleblack,Wesley Jones

2024-02-23

Abstract:Stochastic optimization problems are powerful models for the operation of systems under uncertainty and are in general computationally intensive to solve. Two-stage stochastic optimization is one such problem, where the objective function involves calculating the expected cost of future decisions to inform the best decision in the present. In general, even approximating this expectation value is a #P-Hard problem. We provide a quantum algorithm to estimate the expected value function in a two-stage stochastic optimization problem in time complexity largely independent from the complexity of the random variable. Our algorithm works in two steps: (1) By representing the random variable in a register of qubits and using this register as control logic for a cost Hamiltonian acting on the primary system of qubits, we use the quantum alternating operator ansatz (QAOA) with operator angles following an annealing schedule to converge to the minimal decision for each scenario in parallel. (2) We then use Quantum Amplitude Estimation (QAE) to approximate the expected value function of the per-scenario optimized wavefunction. We show that the annealing time of the procedure in (1) is independent of the number of scenarios in the probability distribution. Additionally, estimation error in QAE converges inverse-linear in the number of "repetitions" of the algorithm, as opposed to converging as the inverse of the square root in traditional Monte Carlo sampling. Because both QAOA and QAE are expected to have polynomial advantage over their classical computing counterparts, we expect our algorithm to have polynomial advantage over classical methods to compute the expected value function. We implement our algorithms for a simple optimization problem inspired by operating the power grid with renewable generation and uncertainty in the weather, and give numerical evidence to support our arguments.

Quantum Physics
Stochastic decomposition for risk-averse two-stage stochastic linear programs

Prasad Parab,Lewis Ntaimo,Bernardo Pagnoncelli

DOI: https://doi.org/10.1007/s10898-024-01432-x

2024-11-02

Journal of Global Optimization

Abstract:Two-stage risk-averse stochastic programming goes beyond the classical expected value framework and aims at controlling the variability of the cost associated with different outcomes based on a choice of a risk measure. In this paper, we study stochastic decomposition (SD) for solving large-scale risk-averse stochastic linear programs with deviation and quantile risk measures. Large-scale problems refer to instances involving too many outcomes to handle using a direct solver, requiring the use of sampling approaches. SD follows an internal sampling approach in which only one sample is randomly generated at each iteration of the algorithm and has been successful for the risk-neutral setting. We extend SD to the risk-averse setting and establish asymptotic convergence of the algorithm to an optimal solution if one exists. A salient feature of the SD algorithm is that the number of samples is not fixed a priori , which allows obtaining good candidate solutions using a relatively small number of samples. We derive two variations of the SD algorithm, one with a single cut (Single-Cut SD) to approximate both the expected recourse function and dispersion statistic, and the other with two separate cuts (Separate-Cut SD). We report on a computational study based on standard test instances to evaluate the empirical performance of the SD algorithms in the risk-averse setting. The study shows that both SD algorithms require a relatively small number of scenarios to converge to an optimal solution. In addition, the comparative performance of the Single-Cut and Separate-Cut SD algorithms is problem-dependent.

mathematics, applied,operations research & management science

A decomposition algorithm for two-stage stochastic programs with approximate rotational invariance

Online Successive Convex Approximation for Two-Stage Stochastic Nonconvex Optimization

A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse Functions

Approximate Truthful Mechanism Design For Two-Dimensional Orthogonal Knapsack Problem

A Decomposition Method for Distributionally-Robust Two-stage Stochastic Mixed-integer Cone Programs

Two-Stage Stochastic Optimization via Primal-Dual Decomposition and Deep Unrolling

Asymptotic Results of Stochastic Decomposition for Two-Stage Stochastic Quadratic Programming

On the Convergence of L-shaped Algorithms for Two-Stage Stochastic Programming

Unified Branch-and-Benders-Cut for two-stage stochastic mixed-integer programs

Towards Global Solutions for Nonconvex Two-Stage Stochastic Programs: A Polynomial Lower Approximation Approach

An Optimal Staging Algorithm

A decomposition algorithm for solving large-scale quadratic programming problems

Solving Two-Stage Stochastic Programs with Endogenous Uncertainty via Random Variable Transformation

Efficient algorithms for Tucker decomposition via approximate matrix multiplication

An Efficient Randomized Algorithm for Computing the Approximate Tucker Decomposition

A randomized operator splitting scheme inspired by stochastic optimization methods

Decomposition Algorithms for Stochastic Programming on a Computational Grid

The decomposition-based outer approximation algorithm for convex mixed-integer nonlinear programming

Calculating the expected value function of a two-stage stochastic optimization program with a quantum algorithm

Stochastic decomposition for risk-averse two-stage stochastic linear programs