Devang Sinha,Siddhartha P. Chakrabarty
Abstract:In this paper, we examine the Sample Average Approximation (SAA) procedure within a framework where the Monte Carlo estimator of the expectation is biased. We also introduce Multilevel Monte Carlo (MLMC) in the SAA setup to enhance the computational efficiency of solving optimization problems. In this context, we conduct a thorough analysis, exploiting Cramér's large deviation theory, to establish uniform convergence, quantify the convergence rate, and determine the sample complexity for both standard Monte Carlo and MLMC paradigms. Additionally, we perform a root-mean-squared error analysis utilizing tools from empirical process theory to derive sample complexity without relying on the finite moment condition typically required for uniform convergence results. Finally, we validate our findings and demonstrate the advantages of the MLMC estimator through numerical examples, estimating Conditional Value-at-Risk (CVaR) in the Geometric Brownian Motion and nested expectation framework.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the convergence and complexity analysis of the Sample Average Approximation (SAA) method in the presence of bias. Specifically, the paper explores how the computational complexity of the SAA process is affected when the Monte Carlo estimator used to approximate the expected value is biased. In addition, the paper also introduces the Multilevel Monte Carlo (MLMC) method to improve the computational efficiency of solving optimization problems, and conducts a detailed analysis of the uniform convergence, convergence rate, and sample complexity under the standard Monte Carlo and MLMC paradigms.
### Main Research Contents
1. **SAA Analysis under the Bias Framework**:
- The paper first conducts a thorough analysis of the SAA method under the bias framework, uses Cramér's large - deviation theory to establish uniform convergence, quantifies the convergence rate, and determines the sample complexity.
- Specifically, the paper considers the following optimization problem:
\[
\min_{x \in X} \{ F(x) := \mathbb{E}[f(x, \zeta)] \}
\]
where \( X \subseteq \mathbb{R}^d \) is a finite - dimensional compact set, \( f(\cdot, \zeta) \) is the cost function, and \(\zeta\) is a random vector.
2. **Application of Multilevel Monte Carlo (MLMC) in SAA**:
- The MLMC method is introduced to enhance the computational efficiency of SAA.
- The multi - level SAA optimization problem is defined as:
\[
\min_{x \in X} \left\{ F_L(x) = \sum_{\ell = 0}^L \frac{1}{N_\ell} \sum_{j = 1}^{N_\ell} \left( f(x, \zeta_j^\ell) - f(x, \zeta_j^{\ell - 1}) \right) \right\}
\]
where \(\zeta^\ell\) is the \(\ell\)-th level approximation of the random variable \(\zeta\).
3. **Root - Mean - Square Error (RMSE) Analysis**:
- Using the tools of empirical process theory, the RMSE analysis of the optimal values under the standard Monte Carlo and MLMC paradigms is carried out.
- The sample complexity required to obtain \(\epsilon\)-RMSE without relying on the finite moment condition is derived.
### Experimental Verification
- The paper verifies its theoretical results through numerical experiments, especially in the application of estimating Conditional Value - at - Risk (CVaR) in the geometric Brownian motion and nested expectation frameworks, showing the advantages of the MLMC estimator.
### Conclusion
- The paper fills the gap in the research of the SAA method under the bias framework and provides a detailed analysis of the convergence and complexity of SAA and MLMC - SAA in the presence of bias.
- The research results show that MLMC - SAA can achieve the same computational complexity as in the unbiased framework in some cases, thus improving the efficiency of solving optimization problems.