On Semi-Stochastic Model for Multi-Stage Decision Making Under Uncertainty

Arkadi Nemirovski

DOI: https://doi.org/10.48550/arXiv.1912.02704

2019-12-07

Abstract:We propose a (seemingly) new computationally tractable model for multi-stage decision making under stochastic uncertainty.

Optimization and Control

What problem does this paper attempt to address?

Multistage Robust Mixed-Integer Optimization under Endogenous Uncertainty

Wei Feng,Yiping Feng,Qi Zhang

DOI: https://doi.org/10.1016/j.ejor.2021.01.048

IF: 6.4

2021-01-01

European Journal of Operational Research

Abstract:Endogenous, i.e. decision-dependent, uncertainty has received increased interest in the stochastic programming community. In the robust optimization context, however, it has rarely been considered. This work addresses multistage robust mixed-integer optimization with decision-dependent uncertainty sets. The proposed framework allows us to consider both continuous and integer recourse, including recourse decisions that affect the uncertainty set. We derive a tractable reformulation of the problem by leveraging recent advances in the construction of nonlinear decision rules, and introduce discontinuous piecewise linear decision rules for continuous recourse. Computational experiments are performed to gain insights on the impact of endogenous uncertainty, the benefit of discrete recourse, and computational performance. Our results indicate that the level of conservatism in the solution can be significantly reduced if endogenous uncertainty and mixed-integer recourse are properly modeled.
Decision-Making Under Uncertainty: Beyond Probabilities

Thom Badings,Thiago D. Simão,Marnix Suilen,Nils Jansen

2023-03-10

Abstract:This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty. The paper features an overview of Markov decision processes (MDPs) and extensions to account for partial observability and adversarial behavior. These models sufficiently capture aleatoric uncertainty but fail to account for epistemic uncertainty robustly. Consequently, we present a thorough overview of so-called uncertainty models that exhibit uncertainty in a more robust interpretation. We show several solution techniques for both discrete and continuous models, ranging from formal verification, over control-based abstractions, to reinforcement learning. As an integral part of this paper, we list and discuss several key challenges that arise when dealing with rich types of uncertainty in a model-based fashion.

Artificial Intelligence,Machine Learning,Robotics,Systems and Control
On Risk-Sensitive Decision Making Under Uncertainty

Chung-Han Hsieh,Yi-Shan Wong

2024-04-20

Abstract:This paper studies a risk-sensitive decision-making problem under uncertainty. It considers a decision-making process that unfolds over a fixed number of stages, in which a decision-maker chooses among multiple alternatives, some of which are deterministic and others are stochastic. The decision-maker's cumulative value is updated at each stage, reflecting the outcomes of the chosen alternatives. After formulating this as a stochastic control problem, we delineate the necessary optimality conditions for it. Two illustrative examples from optimal betting and inventory management are provided to support our theory.

Optimization and Control,Systems and Control,Computational Finance,Methodology
Distributionally Robust Multi-Stage Decision Problem and Satisficing

Chi Xie,Runhao Zhang,Daniel Zhuoyu Long

DOI: https://doi.org/10.2139/ssrn.4535983

2023-01-01

SSRN Electronic Journal

Abstract:We study the classical multi-stage stochastic decision problem, wherein the exact distribution of uncertainty is unknown. We first employ an ambiguity set to characterize the potential distributions and model the problem within the conventional framework of distributionally robust optimization. We demonstrate that solving such a problem is computationally demanding. Then, we introduce the robust satisficing framework, which adopts a target-driven approach to account for all possible probability distributions. Under this framework, we use Kullback-Leibler divergence to measure the distance from potential distributions to the reference distribution. We show that the multi-stage stochastic decision problem can be reformulated into a series of dynamic programming problems, and the optimal policy can be determined through a Bellman-type backward induction. Finally, we apply this method to an inventory management problem and illustrate the optimality of base-stock policy under the robust satisficing framework. Through our numerical results, we show the promising performance of our approach in effectively addressing uncertain scenarios and achieving predefined target.
An Anytime Algorithm for Decision Making under Uncertainty

Michael C. Horsch,David L. Poole

DOI: https://doi.org/10.48550/arXiv.1301.7384

2013-01-30

Abstract:We present an anytime algorithm which computes policies for decision problems represented as multi-stage influence diagrams. Our algorithm constructs policies incrementally, starting from a policy which makes no use of the available information. The incremental process constructs policies which includes more of the information available to the decision maker at each step. While the process converges to the optimal policy, our approach is designed for situations in which computing the optimal policy is infeasible. We provide examples of the process on several large decision problems, showing that, for these examples, the process constructs valuable (but sub-optimal) policies before the optimal policy would be available by traditional methods.

Artificial Intelligence
Solving Two-Stage Stochastic Programs with Endogenous Uncertainty via Random Variable Transformation

Maria Bazotte,Margarida Carvalho,Thibaut Vidal

2024-07-18

Abstract:Real-world decision-making problems often involve decision-dependent uncertainty, where the probability distribution of the random vector depends on the model decisions. Few studies focus on two-stage stochastic programs with this type of endogenous uncertainty, and those that do lack general methodologies. We propose a general method for solving a class of these programs based on random variable transformation, a technique widely employed in probability and statistics. The random variable transformation converts a stochastic program with endogenous uncertainty (original program) into an equivalent stochastic program with decision-independent uncertainty (transformed program), for which solution procedures are well-studied. Additionally, endogenous uncertainty usually leads to nonlinear nonconvex programs, which are theoretically intractable. Nonetheless, we show that, for some classical endogenous distributions, the proposed method yields mixed-integer linear or convex programs with exogenous uncertainty. We validate this method by applying it to a network design and facility-protection problem, considering distinct decision-dependent distributions for the random variables. While the original formulation of this problem is nonlinear nonconvex for most endogenous distributions, the proposed method transforms it into mixed-integer linear programs with exogenous uncertainty. We solve these transformed programs with the sample average approximation method. We highlight the superior performance of our approach compared to solving the original program in the case a mixed-integer linear formulation of this program exists.

Optimization and Control
A multi-stage anticipated surprise model with dynamic expectation for economic decision-making

Ho Ka Chan,Taro Toyoizumi

DOI: https://doi.org/10.1038/s41598-023-50529-y

IF: 4.6

2024-01-06

Scientific Reports

Abstract:There are many modeling works that aim to explain people's behaviors that violate classical economic theories. However, these models often do not take into full account the multi-stage nature of real-life problems and people's tendency in solving complicated problems sequentially. In this work, we propose a descriptive decision-making model for multi-stage problems with perceived post-decision information. In the model, decisions are chosen based on an entity which we call the 'anticipated surprise'. The reference point is determined by the expected value of the possible outcomes, which we assume to be dynamically changing during the mental simulation of a sequence of events. We illustrate how our formalism can help us understand prominent economic paradoxes and gambling behaviors that involve multi-stage or sequential planning. We also discuss how neuroscience findings, like prediction error signals and introspective neuronal replay, as well as psychological theories like affective forecasting, are related to the features in our model. This provides hints for future experiments to investigate the role of these entities in decision-making.

multidisciplinary sciences
A stochastic programming model for money management

Bennett Golub,Martin Holmer,Raymond McKendall,Lawrence Pohlman,Stavros A. Zenios

DOI: https://doi.org/10.1016/0377-2217(94)00038-e

IF: 6.4

1995-09-01

European Journal of Operational Research

Abstract:Portfolio managers in the new fixed-income securities have to cope with various forms of uncertainty, in addition to the usual interest rate changes. Uncertainy in the timing and amount of cashflows, changes in the default and other risk premia and so on, complicate the portfolio manager's problem. We develop here a multi-period, dynamic, portfolio optimization model to address this problem. The model specifies a sequence of investment decisions over time that maximize the expected utility of return at the end of the planning horizon. The model is a two-stage stochastic program with recourse. The dynamics of interest rates, cashflow uncertainty, and liquidity, default and other risk premia, are explicitly modeled through postulated scenarios. Simulation procedures are developed to generate these scenarios. The optimization models are then integrated with the simulation procedures. Extensive validation experiments are carried out to establish the effectiveness of the model in dealing with uncertainty. In particular the model is compared against the popular portfolio immunization strategy, and against a portfolio based on mean-absolute deviation optimization.

operations research & management science
Adaptive Multistage Stochastic Programming

Sezen Ece Kayacık,Beste Basciftci,Albert H Schrotenboer,Evrim Ursavas

2024-01-15

Abstract:Multistage stochastic programming is a powerful tool allowing decision-makers to revise their decisions at each stage based on the realized uncertainty. However, in practice, organizations are not able to be fully flexible, as decisions cannot be revised too frequently due to their high organizational impact. Consequently, decision commitment becomes crucial to ensure that initially made decisions remain unchanged for a certain period. This paper introduces adaptive multistage stochastic programming, a new optimization paradigm that strikes an optimal balance between decision flexibility and commitment by determining the best stages to revise decisions depending on the allowed level of flexibility. We introduce a novel mathematical formulation and theoretical properties eliminating certain constraint sets. Furthermore, we develop a decomposition method that effectively handles mixed-integer adaptive multistage programs by adapting the integer L-shaped method and Benders decomposition. Computational experiments on stochastic lot-sizing and generation expansion planning problems show substantial advantages attained through optimal selections of revision times when flexibility is limited, while demonstrating computational efficiency of the proposed properties and solution methodology. Optimizing revision times in a less flexible case can outperform arbitrary selection in a more flexible case. By adhering to these optimal revision times, organizations can achieve performance levels comparable to fully flexible settings.

Optimization and Control
A Note on Piecewise Affine Decision Rules for Robust, Stochastic, and Data-Driven Optimization

Simon Thomä,Maximilian Schiffer,Wolfram Wiesemann

2024-09-16

Abstract:Multi-stage decision-making under uncertainty, where decisions are taken under sequentially revealing uncertain problem parameters, is often essential to faithfully model managerial problems. Given the significant computational challenges involved, these problems are typically solved approximately. This short note introduces an algorithmic framework that revisits a popular approximation scheme for multi-stage stochastic programs by Georghiou et al. (2015) and improves upon it to deliver superior policies in the stochastic setting, as well as extend its applicability to robust optimization and a contemporary Wasserstein-based data-driven setting. We demonstrate how the policies of our framework can be computed efficiently, and we present numerical experiments that highlight the benefits of our method.

Optimization and Control
A dynamic stochastic programming model for international portfolio management

Nikolas Topaloglou,Hercules Vladimirou,Stavros A. Zenios

DOI: https://doi.org/10.1016/j.ejor.2005.07.035

IF: 6.4

2008-03-01

European Journal of Operational Research

Abstract:We develop a multi-stage stochastic programming model for international portfolio management in a dynamic setting. We model uncertainty in asset prices and exchange rates in terms of scenario trees that reflect the empirical distributions implied by market data. The model takes a holistic view of the problem. It considers portfolio rebalancing decisions over multiple periods in accordance with the contingencies of the scenario tree. The solution jointly determines capital allocations to international markets, the selection of assets within each market, and appropriate currency hedging levels. We investigate the performance of alternative hedging strategies through extensive numerical tests with real market data. We show that appropriate selection of currency forward contracts materially reduces risk in international portfolios. We further find that multi-stage models consistently outperform single-stage models. Our results demonstrate that the stochastic programming framework provides a flexible and effective decision support tool for international portfolio management.

operations research & management science
The Value of Multi-Stage Stochastic Programming in Risk-Averse Unit Commitment Under Uncertainty

Ali rfan Mahmutoullar,Shabbir Ahmed,zlem avu,M. Selim Aktrk,Ali Irfan Mahmutogullari,Ozlem Cavus,M. Selim Akturk

DOI: https://doi.org/10.1109/tpwrs.2019.2902511

IF: 7.326

2019-09-01

IEEE Transactions on Power Systems

Abstract:Day-ahead scheduling of electricity generation or unit commitment is an important and challenging optimization problem in power systems. Variability in net load arising from the increasing penetration of renewable technologies has motivated study of various classes of stochastic unit commitment models. In two-stage models, the generation schedule for the entire day is fixed while the dispatch is adapted to the uncertainty, whereas in multi-stage models the generation schedule is also allowed to dynamically adapt to the uncertainty realization. Multi-stage models provide more flexibility in the generation schedule; however, they require significantly higher computational effort than two-stage models. To justify this additional computational effort, we provide theoretical and empirical analyses of the value of multi-stage solution for risk-averse multi-stage stochastic unit commitment models. The value of multi-stage solution measures the relative advantage of multi-stage solutions over their two-stage counterparts. Our results indicate that, for unit commitment models, the value of multi-stage solution increases with the level of uncertainty and number of periods, and decreases with the degree of risk aversion of the decision maker.

engineering, electrical & electronic
Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets

Xian Yu,Siqian Shen

DOI: https://doi.org/10.1007/s10107-020-01580-4

IF: 2.7

2020-10-21

Mathematical Programming

Abstract:We study multistage distributionally robust mixed-integer programs under endogenous uncertainty, where the probability distribution of stage-wise uncertainty depends on the decisions made in previous stages. We first consider two ambiguity sets defined by decision-dependent bounds on the first and second moments of uncertain parameters and by mean and covariance matrix that exactly match decision-dependent empirical ones, respectively. For both sets, we show that the subproblem in each stage can be recast as a mixed-integer linear program (MILP). Moreover, we extend the general moment-based ambiguity set in Delage and Ye (Oper Res 58(3):595–612, 2010) to the multistage decision-dependent setting, and derive mixed-integer semidefinite programming (MISDP) reformulations of stage-wise subproblems. We develop methods for attaining lower and upper bounds of the optimal objective value of the multistage MISDPs, and approximate them using a series of MILPs. We deploy the Stochastic Dual Dynamic integer Programming (SDDiP) method for solving the problem under the three ambiguity sets with risk-neutral or risk-averse objective functions, and conduct numerical studies on multistage facility-location instances having diverse sizes under different parameter and uncertainty settings. Our results show that the SDDiP quickly finds optimal solutions for moderate-sized instances under the first two ambiguity sets, and also finds good approximate bounds for the multistage MISDPs derived under the third ambiguity set. We also demonstrate the efficacy of incorporating decision-dependent distributional ambiguity in multistage decision-making processes.

mathematics, applied,computer science, software engineering,operations research & management science
OPTIMAL ALTERNATIVE SELECTION MODELS IN A MULTI-STAGE DECISION-MAKING PROCESS

Oksana Mulesa,Vitaliy Snytyuk,Ivan Myronyuk

DOI: https://doi.org/10.21303/2461-4262.2019.001005

2019-11-30

Abstract:Management decision-making tasks are usually characterized by a high level of uncertainty. When solving this class of problems, it is necessary to take into account the environmental conditions for the implementation of the decisions made and the consequences that may arise in this case. The decision-making task in the face of uncertainty can be represented in the form of a “game with nature”, in which the optimal player strategy is sought. A two-stage decision-making process is considered, in which at each stage the decision-making problem is solved in conditions of risk. The case is supposed in which, after making a decision at the first stage, choosing an effective alternative and the onset of a certain state of nature, it is necessary to solve the decision-making problem of the second stage. Decision-making models based on well-known decision models of the “game with nature” are proposed. The developed models allow in the process of choosing an effective alternative to the first stage to assess the possible consequences of such a choice, taking into account the expectations of the decision maker. In the course of experimental verification, it is shown that the developed decision-making models can be used to solve such multi-stage problems, the phased solution of which is incorrect. This may occur due to the fact that some of their stages are associated with certain losses, and others – with profit. In such situations, it is advisable to consider the problem as a whole and at each stage, take into account all available information as much as possible.
A Node Formulation for Multistage Stochastic Programs with Endogenous Uncertainty

Giovanni Pantuso

DOI: https://doi.org/10.1007/s10287-021-00390-z

2021-02-14

Abstract:This paper introduces a node formulation for multistage stochastic programs with endogenous (i.e., decision-dependent) uncertainty. Problems with such structure arise when the choices of the decision maker determine a change in the likelihood of future random events. The node formulation avoids an explicit statement of non-anticipativity constraints, and as such keeps the dimension of the model sizeable. An exact solution algorithm for a special case is introduced and tested on a case study. Results show that the algorithm outperforms a commercial solver as the size of the instances increases.

Optimization and Control
A Successive Refinement for Solving Stochastic Programs with Decision-Dependent Random Capacities

Hugh Medal,Samuel Affar

2024-09-13

Abstract:We study a class of two-stage stochastic programs in which the second stage includes a set of components with uncertain capacity, and the expression for the distribution function of the uncertain capacity includes first-stage variables. Thus, this class of problems has the characteristics of a stochastic program with decision-dependent uncertainty. A natural way to formulate this class of problems is to enumerate the scenarios and express the probability of each scenario as a product of the first-stage decision variables; unfortunately, this formulation results in an intractable model with a large number of variable products with high-degree. After identifying structural results related to upper and lower bounds and how to improve these bounds, we present a successive refinement algorithm that successively and dynamically tightens these bounds. Implementing the algorithm within a branch-and-cut method, we report the results of computational experiments that indicate that the successive refinement algorithm significantly outperforms a benchmark approach. Specifically, results show that the algorithm finds an optimal solution before the refined state space become too large.

Optimization and Control
Evolutionary Optimization for Decision Making under Uncertainty

Ronald Hochreiter

DOI: https://doi.org/10.48550/arXiv.1401.4696

2014-01-20

Abstract:Optimizing decision problems under uncertainty can be done using a variety of solution methods. Soft computing and heuristic approaches tend to be powerful for solving such problems. In this overview article, we survey Evolutionary Optimization techniques to solve Stochastic Programming problems - both for the single-stage and multi-stage case.

Neural and Evolutionary Computing
Discrete time optimal investment under model uncertainty

Laurence Carassus,Massinissa Ferhoune

2024-02-28

Abstract:We study a robust utility maximization problem in a general discrete-time frictionless market under quasi-sure no-arbitrage. The investor is assumed to have a random and concave utility function defined on the whole real-line. She also faces model ambiguity on her beliefs about the market, which is modeled through a set of priors. We prove the existence of an optimal investment strategy using only primal methods. For that we assume classical assumptions on the market and on the random utility function as asymptotic elasticity constraints. Most of our other assumptions are stated on a prior-by-prior basis and correspond to generally accepted assumptions in the literature on markets without ambiguity. We also propose a general setting including utility functions with benchmark for which our assumptions are easily checked.

Mathematical Finance,Portfolio Management
From Data to Stochastic Modeling and Decision Making: What Can We Do Better?

Joost Berkhout,Bernd Heidergott,Henry Lam,Yijie Peng

DOI: https://doi.org/10.1142/s0217595919400128

2019-01-01

Asia-Pacific Journal of Operational Research

Abstract:In the past decades we have witnessed a paradigm-shift from scarcity of data to abundance of data. Big data and data analytics have fundamentally reshaped many areas including operations research. In this paper, we discuss how to integrate data with the model-based analysis in a controlled way. Specifically, we consider techniques to quantify input uncertainty and the decision making under input uncertainty. Numerical experiments demonstrate that different ways in decision making may lead to significantly different outcomes in a maintenance problem.

On Semi-Stochastic Model for Multi-Stage Decision Making Under Uncertainty

Multistage Robust Mixed-Integer Optimization under Endogenous Uncertainty

Decision-Making Under Uncertainty: Beyond Probabilities

On Risk-Sensitive Decision Making Under Uncertainty

Distributionally Robust Multi-Stage Decision Problem and Satisficing

An Anytime Algorithm for Decision Making under Uncertainty

Solving Two-Stage Stochastic Programs with Endogenous Uncertainty via Random Variable Transformation

A multi-stage anticipated surprise model with dynamic expectation for economic decision-making

A stochastic programming model for money management

Adaptive Multistage Stochastic Programming

A Note on Piecewise Affine Decision Rules for Robust, Stochastic, and Data-Driven Optimization

A dynamic stochastic programming model for international portfolio management

The Value of Multi-Stage Stochastic Programming in Risk-Averse Unit Commitment Under Uncertainty

Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets

OPTIMAL ALTERNATIVE SELECTION MODELS IN A MULTI-STAGE DECISION-MAKING PROCESS

A Node Formulation for Multistage Stochastic Programs with Endogenous Uncertainty

A Successive Refinement for Solving Stochastic Programs with Decision-Dependent Random Capacities

Evolutionary Optimization for Decision Making under Uncertainty

Discrete time optimal investment under model uncertainty

From Data to Stochastic Modeling and Decision Making: What Can We Do Better?