A Unified Framework for Optimization under Uncertainty TutORials in Operations Research

Warrren B Powell

Abstract:Stochastic optimization, also known as optimization under uncertainty, is studied by over a dozen communities, often (but not always) with different notational systems and styles, typically motivated by different problem classes (or sometimes different research questions) which often lead to different algorithmic strategies. This resulting “jungle of stochastic optimization” has produced a highly fragmented set of research communities which complicates the sharing of ideas. This tutorial unifies the modeling of a wide range of problems, from dynamic programming to stochastic programming to multiarmed bandit problems to optimal control, in a common mathematical framework that is centered on the search for policies. We then identify two fundamental strategies for finding effective policies, which leads to four fundamental classes of policies which span every field of research in stochastic optimization.