Abstract:This study focuses on two important problems related to applying offline model-based optimization to real-world industrial control problems. The first problem is how to create a reliable probabilistic model that accurately captures the dynamics present in noisy industrial data. The second problem is how to reliably optimize control parameters without actively collecting feedback from industrial systems. Specifically, we introduce a novel cGAN ensemble-based uncertainty-aware surrogate model for reliable offline model-based optimization in industrial control problems. The effectiveness of the proposed method is demonstrated through extensive experiments conducted on two representative cases, namely a discrete control case and a continuous control case. The results of these experiments show that our method outperforms several competitive baselines in the field of offline model-based optimization for industrial control.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper mainly focuses on two important challenges in applying offline model - based optimization methods to practical industrial control problems:
1. **How to create a reliable probability model**: This model needs to be able to accurately capture the dynamic characteristics in noisy industrial data. There are usually a large amount of noise in industrial systems, which may come from partially observable states, imperfect sensor measurements, and actuator controls, etc. Therefore, constructing a model that can accurately reflect this complexity and uncertainty is a key issue.
2. **How to reliably optimize control parameters without actively collecting feedback**: Since the experimental costs of industrial systems are often very high, it is necessary to learn from the existing offline data and optimize the control strategy without further data collection. This requires that the learned control strategy can still maintain reliability when facing unseen data.
To solve these problems, the author proposes a new uncertainty - aware surrogate model integrated with conditional generative adversarial networks (cGAN). This model aims to improve the reliability and performance of offline model - based optimization in industrial control problems in the following ways:
- **Modeling aleatoric uncertainty**: Minimize assumptions about hidden dynamics through cGAN integration to more accurately capture the stochastic uncertainty in the system.
- **Introducing an uncertainty - aware reward - penalty mechanism**: Ensure that overly high reward estimates are not given to out - of - distribution (OoD) inputs during the optimization process, especially in cases of high prediction uncertainty.
Through extensive experimental verification, this method performs well in both discrete control and continuous control typical scenarios, significantly outperforming other competitive baseline methods.
#### Summary of Mathematical Formulas
1. **Expected Reward Maximization Problem (Discrete Control)**:
\[
\max_{\pi} \mathbb{E}_{x,u \sim \pi} \left[ \int p(y|x,u) r(y) dy \right]
\]
2. **Expected Reward Maximization Problem (Continuous Control)**:
\[
\max_{\pi} \mathbb{E}_{x_0:T, u_0:T \sim \pi} \left[ \sum_{t = 0}^T \gamma^t \int p(y_t|x_t, u_t) r(y_t) dy_t \right]
\]
where $\gamma\in(0,1)$ is the discount factor for future rewards, and $T$ is a finite or infinite time horizon.
3. **Optimization Problem of Generative Adversarial Networks (GAN)**:
\[
\min_{\theta_G} \max_{\theta_D} \mathbb{E}_{x \sim P_{data}(x)} [\log D(x; \theta_D)]+ \mathbb{E}_{z \sim P(z)} [\log (1 - D(G(z; \theta_G)))]
\]
4. **Uncertainty Quantification of Deep Ensemble**:
\[
U(x)\approx H[P(y|x, \Theta)]-\frac{1}{M} \sum_{m = 1}^M H[P(y|x, \theta^{(m)})]+\frac{1}{M} \sum_{m = 1}^M H[P(y|x, \theta^{(m)})]
\]
where $H[\cdot]$ represents entropy, $U(x)$ represents the total uncertainty at input $x$, which is decomposed into epistemic uncertainty and aleatoric uncertainty.
5. **Uncertainty - Aware Reward - Penalty Mechanism**:
\[
r_p(x, u)=
\begin{cases}
\end{cases}