Abstract:In this paper, we consider the alignment between an upstream dimensionality reduction task of learning a low-dimensional representation of a set of high-dimensional data and a downstream optimization task of solving a stochastic program parameterized by said representation. In this case, standard dimensionality reduction methods (e.g., principal component analysis) may not perform well, as they aim to maximize the amount of information retained in the representation and do not generally reflect the importance of such information in the downstream optimization problem. To address this problem, we develop a prescriptive dimensionality reduction framework that aims to minimize the degree of suboptimality in the optimization phase. For the case where the downstream stochastic optimization problem has an expected value objective, we show that prescriptive dimensionality reduction can be performed via solving a distributionally-robust optimization problem, which admits a semidefinite programming relaxation. Computational experiments based on a warehouse transshipment problem and a vehicle repositioning problem show that our approach significantly outperforms principal component analysis with real and synthetic data sets.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
The paper aims to solve the problem of how to provide optimal decisions for downstream stochastic optimization problems through dimension - reduction techniques under high - dimensional data input. Specifically, the paper focuses on a two - consecutive - stage process:
1. **Dimension - reduction stage**: Learn low - dimensional representations from high - dimensional data, that is, find a low - dimensional space that can describe the distribution of high - dimensional data.
2. **Stochastic optimization stage**: Based on the above low - dimensional representation, solve a parameterized stochastic optimization problem, where the uncertainty of parameters is controlled by the low - dimensional representation.
### Limitations of standard dimension - reduction methods
Standard dimension - reduction methods (such as Principal Component Analysis, PCA) usually aim to maximize the amount of information retention in low - dimensional representations, but these methods do not consider the importance of this information in downstream optimization problems. Therefore, standard dimension - reduction methods may not be able to identify factors that are crucial for downstream stochastic optimization problems, resulting in sub - optimal solutions.
### Solutions
To solve this problem, the paper proposes a framework called "Prescriptive Principal Component Analysis" (Prescriptive PCA, PPCA). The goal of this framework is to minimize the sub - optimality in the downstream optimization stage. Specifically, the paper shows how to achieve prescriptive dimension - reduction by solving a distributionally robust optimization problem, which can be solved by semidefinite programming relaxation.
### Main contributions
1. **Problem identification**: The paper points out the limitations of standard dimension - reduction methods (such as PCA) in two - stage problems, especially their inability to identify factors important for downstream stochastic optimization problems.
2. **Method proposal**: Proposes the PPCA framework to learn low - dimensional representations by minimizing the sub - optimality of downstream optimization problems.
3. **Theoretical analysis**: Proves that the surrogate (upper bound) of minimizing sub - optimality can be formulated as a biconvex problem and solved by an alternating algorithm.
4. **Experimental verification**: Through computational experiments on the warehouse transfer problem and the vehicle re - positioning problem, shows the effectiveness of the PPCA method on real - world datasets, which is significantly better than the traditional PCA method.
### Formula summary
- **Second - stage objective function**:
\[
h(z)=\min_y b^T y \quad \text{s.t.} \quad A y \geq z
\]
- **Projected second - stage objective function**:
\[
h(\hat{z}) = h(V V^T z)=\max_{w \geq 0} w^T V V^T z \quad \text{s.t.} \quad w \in P
\]
where \(P = \{w \geq 0\mid A^T w = b\}\).
- **Upper bound of performance loss**:
\[
L(V)=\left\|E[h(\tilde{z})]-E[h(V V^T \tilde{z})]\right\|
\]
- **Distributionally robust optimization problem**:
\[
\bar{h}(\mu,\Sigma)=\max_{p, Y, X}\text{tr}(Y) \quad \text{s.t.} \quad A^T p = b,\quad \text{diag}(A^T X A)=b^2,\quad \begin{pmatrix}1 & \mu^T \\ \mu & \Sigma\end{pmatrix}\succeq 0,\quad p, X \geq 0
\]
- **Optimization form of the PPCA problem**:
\[
\min_{\Sigma_1,\Sigma_e}\bar{h}(0,\Sigma_e)+\theta\langle\Sigma_1,\Sigma_e\rangle \quad \text{s.t.} \quad \Sigma_1+\Sigma_e=\Sigma_0,\quad \Sigma_1,\Sigma_e\succeq 0,\quad