Abstract:Computing cross-partial derivatives using fewer model runs is relevant in modeling, such as stochastic approximation, derivative-based ANOVA, exploring complex models, and active subspaces. This paper introduces surrogates of all the cross-partial derivatives of functions by evaluating such functions at $N$ randomized points and using a set of $L$ constraints. Randomized points rely on independent, central, and symmetric variables. The associated estimators, based on $NL$ model runs, reach the optimal rates of convergence (i.e., $\mathcal{O}(N^{-1})$), and the biases of our approximations do not suffer from the curse of dimensionality for a wide class of functions. Such results are used for i) computing the main and upper-bounds of sensitivity indices, and ii) deriving emulators of simulators or surrogates of functions thanks to the derivative-based ANOVA. Simulations are presented to show the accuracy of our emulators and estimators of sensitivity indices. The plug-in estimates of indices using the U-statistics of one sample are numerically much stable.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to estimate cross - partial derivatives efficiently and accurately in complex computational models. Specifically, the paper proposes a method of constructing surrogate models for all cross - partial derivatives by evaluating the values of functions at random points. These methods can achieve the optimal convergence rate, and for a wide class of functions, their biases are not affected by the curse of dimensionality. The solutions to these problems are of great significance for the following aspects: 1. **Sensitivity analysis**: By calculating the main sensitivity indices and their upper bounds, the input variables and their interactions in high - dimensional models can be effectively screened. 2. **Construction of emulators**: Using derivative - based analysis of variance (Db - ANOVA), emulators or surrogate models of functions can be constructed, thus approximating complex or computationally expensive emulators more quickly. 3. **Active subspaces**: Dimension reduction is carried out through first - order derivatives, and then complex models are approximated. ### Main contributions of the paper 1. **New expressions**: The paper derives new expressions for cross - partial derivatives of any order. These expressions combine the generalized Richardson extrapolation method to improve the accuracy of approximation and the Monte Carlo method based only on independently and symmetrically distributed random variables. 2. **Unbiased upper bounds**: For a wide class of functions, estimates of unbiased upper bounds for cross - partial derivative approximations are provided, and these upper bounds are independent of dimension. 3. **Optimal convergence rate**: The proposed estimators achieve the optimal parametric convergence rate. 4. **Application examples**: The accuracy of the method is demonstrated through simulation experiments of test functions, including the Ishigami function and Sobol's g - function. ### Key formulas - **Expression for cross - partial derivatives**: \[ D^{|u|}f(x)=\sum_{\ell = 1}^L C^{(|u|)}_{\ell}\mathbb{E}\left[f(x+\beta_{\ell}hV)\prod_{k\in u}V_k\left(\frac{h_k}{\sigma^2}\right)\right]+O\left(\|h\|_2^{2\alpha_{|u|}}\right) \] - **Upper bound of bias**: \[ \left|D^{|u|}f(x)-\sum_{\ell = 1}^L C^{(|u|)}_{\ell}\mathbb{E}\left[f(x+\beta_{\ell}hV)\prod_{k\in u}V_k\left(\frac{h_k}{\sigma^2}\right)\right]\right|\leq\sigma^{2L}M_{|u| + 2L}K_{1,L}\left\|h^2\right\|_2^L \] - **Upper bound of mean - square error**: \[ \mathbb{E}\left[(\hat{D}^{|u|}f(x)-D^{|u|}f(x))^2\right]\leq\sigma^2M_{|u|+1}^2K_1^2\Gamma_{|u|+1}^2\|h\|_2^2+\frac{M_{r^*+1}^2\Gamma_{r^*+1}^2K_{2,r^*+1}}{N}\sigma^2(|u|-r^*-1)\prod_{k\in u}h_k^2\left\|h^2\right\|_2^{r^*+1} \] ### Application examples - **Ishigami function**: \[ f(x)=\sin(x_1)+7\sin^2(x_2)+0.1x_3^4\sin(x_1) \] Its sensitivity indices are $S_1 = 0.3139$, $S_2 = 0.4424$, $S_3 = 0.0$, $ST_1 = 0.567$

Optimal estimators of cross-partial derivatives and surrogates of functions

Nonparametric Estimation via Partial Derivatives

On the role of surrogates in the efficient estimation of treatment effects with limited outcome data

Conservative Surrogate Models for Optimization with the Active Subspace Method

Optimal Estimation of Derivatives in Nonparametric Regression

Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations

Towards Sobolev Pruning

Adaptive construction of surrogates for the Bayesian solution of inverse problems

On Parametric Optimal Execution and Machine Learning Surrogates

Nuisance Function Tuning and Sample Splitting for Optimal Doubly Robust Estimation

Uses of Sub-sample Estimates to Reduce Errors in Stochastic Optimization Models

Efficient estimation of divergence-based sensitivity indices with Gaussian process surrogates

General-Surrogate Adaptive Sampling Using Interquartile Range for Design Space Exploration

Surrogate-Based Simulation Optimization

Estimating Treatment Effects using Multiple Surrogates: The Role of the Surrogate Score and the Surrogate Index

Surrogate Active Subspaces for Jump-Discontinuous Functions

Cross-Fitting and Fast Remainder Rates for Semiparametric Estimation

A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

Regularizing nested Monte Carlo Sobol' index estimators to balance the trade-off between explorations and repetitions in global sensitivity analysis of stochastic models

Optimal smoothing parameter selection in single-index model derivative estimation