Abstract:The increasing use of stochastic models for describing complex phenomena warrants surrogate models that capture the reference model characteristics at a fraction of the computational cost, foregoing potentially expensive Monte Carlo simulation. The predominant approach of fitting a large neural network and then pruning it to a reduced size has commonly neglected shortcomings. The produced surrogate models often will not capture the sensitivities and uncertainties inherent in the original model. In particular, (higher-order) derivative information of such surrogates could differ drastically. Given a large enough network, we expect this derivative information to match. However, the pruned model will almost certainly not share this behavior. In this paper, we propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the pruned neural network based surrogate model. We experimentally underpin the method on an example of pricing a multidimensional Basket option modelled through a stochastic differential equation with Brownian motion. The proposed method is, however, not limited to the domain of quantitative finance, which was chosen as a case study for intuitive interpretations of the sensitivities. It serves as a foundation for building further surrogate modelling techniques considering sensitivity information.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of how to retain the sensitivity and uncertainty information of the original complex model when constructing a neural network (NN) surrogate model. Specifically, the paper focuses on incorporating sensitivity information during the training and pruning processes to ensure that the pruned neural network surrogate model can accurately capture the sensitivity and derivative information of the original model. #### Main problem background 1. **Deficiencies of existing methods**: - Existing methods usually train a large neural network first and then reduce it to a smaller model through pruning. However, this method often ignores the sensitivity and uncertainty in the original model. - The pruned model often fails to well recover the high - order derivative information of the original model, which leads to poor performance of the model in some application scenarios, especially in fields that require accurate prediction of sensitivity (such as financial pricing, engineering design, etc.). 2. **Research motivation**: - In many scientific and engineering fields, high - dimensional stochastic models are necessary tools for describing complex phenomena. For example, in biology, materials science, and quantitative finance, it is very common to use stochastic processes for simulation. - Although Monte Carlo (MC) simulation can handle these stochastic models, the computational cost is very high. Therefore, it is particularly important to find surrogate models that can be executed quickly and maintain sufficient accuracy. #### Solutions proposed in the paper To overcome the above problems, the paper proposes the following methods: 1. **Pruning based on Interval Adjoint Significance Analysis (IASA)**: - Use IASA to evaluate the importance of input and intermediate variables and prune according to their importance. This method not only considers the sensitivity of parameters but also ensures that the pruned model can better retain the characteristics of the original model. 2. **Sobolev Training**: - Apply Sobolev Training for fine - tuning on the pruned model. Sobolev Training introduces derivative information into the loss function, enabling the model to better recover the prediction accuracy of the first - order and second - order derivatives. 3. **Combining the advantages of both**: - Combine IASA and Sobolev Training to ensure that the pruned neural network surrogate model is not only smaller in size but also can accurately capture the sensitivity and derivative information of the original model. #### Specific case study The paper takes the Gaussian basket option pricing model as an example for experimental verification. This model has the following characteristics: - **Bachelier model**: Describes the stochastic differential equation (SDE) of asset prices and can be simplified by forward prices. - **Sensitivity analysis**: Includes first - order (Delta) and second - order (Gamma) sensitivities, which are crucial for risk assessment in the financial field. Through experiments, the paper shows that the proposed method can significantly reduce the computational cost of the model while maintaining high accuracy and can better recover the sensitivity information of the original model. ### Summary In general, the main contribution of this paper is to systematically explore how to incorporate sensitivity information during the training and pruning processes, thereby constructing more accurate and efficient neural network surrogate models. This method is not only applicable to option pricing in the financial field but can also be extended to other fields that require accurate prediction of sensitivity.

Towards Sobolev Pruning

Surrogate-Based Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Structural Error.

On Parametric Optimal Execution and Machine Learning Surrogates

Surrogate-assisted global sensitivity analysis: an overview

Beyond Surrogate Modeling: Learning the Local Volatility Via Shape Constraints

Sensitivity-Aware Amortized Bayesian Inference

Probabilistic Surrogate Networks for Simulators with Unbounded Randomness

Conservative Surrogate Models for Optimization with the Active Subspace Method

Advances in surrogate based modeling, feasibility analysis, and optimization: A review

Statistical Machine Learning for Quantitative Finance

A framework for strategic discovery of credible neural network surrogate models under uncertainty

Efficient Learning of Accurate Surrogates for Simulations of Complex Systems

Optimal estimators of cross-partial derivatives and surrogates of functions

Multi-fidelity reduced-order surrogate modelling

Goal-oriented adaptive surrogate construction for stochastic inversion

Surrogate-Based Simulation Optimization

Surrogate Modeling for Physical Systems with Preserved Properties and Adjustable Tradeoffs

Sobolev neural network with residual weighting as a surrogate in linear and non-linear mechanics

A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations