Abstract:Surrogate models are frequently used to replace costly engineering simulations. A single surrogate is frequently chosen based on previous experience or by fitting multiple surrogates and selecting one based on mean cross-validation errors. A novel stacking strategy will be presented in this paper. This new strategy results from reinterpreting the model selection process based on the generalization error. For the first time, this problem is proposed to be translated into a well-studied financial problem: portfolio management and optimization. In short, it is demonstrated that the individual residues calculated by leave-one-out procedures are samples from a given random variable ϵi, whose second non-central moment is the i-th model's generalization error. Thus, a stacking methodology based solely on evaluating the behavior of the linear combination of the random variables ϵi is proposed. At first, several surrogate models are calibrated. The Directed Bubble Hierarchical Tree (DBHT) clustering algorithm is then used to determine which models are worth stacking. The stacking weights can be calculated using any financial approach to the portfolio optimization problem. This alternative understanding of the problem enables practitioners to use established financial methodologies to calculate the models' weights, significantly improving the ensemble of models' out-of-sample performance. A study case is carried out to demonstrate the applicability of the new methodology. Overall, a total of 124 models were trained using a specific dataset: 40 Machine Learning models and 84 Polynomial Chaos Expansion models (which considered 3 types of base random variables, 7 least square algorithms for fitting the up to fourth order expansion's coefficients). Among those, 99 models could be fitted without convergence and other numerical issues. The DBHT algorithm with Pearson correlation distance and generalization error similarity was able to select a subgroup of 23 models from the 99 fitted ones, implying a reduction of about 77% in the total number of models, representing a good filtering scheme which still preserves diversity. Finally, it has been demonstrated that the weights obtained by building a Hierarchical Risk Parity (HPR) portfolio perform better for various input random variables, indicating better out-of-sample performance. In this way, an economic stacking strategy has demonstrated its worth in improving the out-of-sample capabilities of stacked models, which illustrates how the new understanding of model stacking methodologies may be useful.

Stochastic reserving with a stacked model based on a hybridized Artificial Neural Network

Forecasting volatility with a stacked model based on a hybridized Artificial Neural Network

Mack-Net model: Blending Mack's model with Recurrent Neural Networks

Stochastic Loss Reserving: Dependence and Estimation

Ensemble distributional forecasting for insurance loss reserving

A hierarchical reserving model for reported non-life insurance claims

Bayesian Non-linear Hierarchical Models for Claims Reserving

Stochastic Claims Reserving In General Insurance: Models And Methodoledgies

Modeling Credit Risk with a Multi‐stage Hybrid Model: an Alternative Statistical Approach

Recurrent Neural Networks for Multivariate Loss Reserving and Risk Capital Analysis

Comparison of Stochastic Claims Reserving Models in Insurance

Hybrid Model for Credit Risk Prediction: an Application of Neural Network Approaches.

Micro-level Reserving for General Insurance Claims using a Long Short-Term Memory Network

Stochastic Loss Reserving in Discrete Time: Individual Vs. Aggregate Data Models

Forecasting reserve risk for temporal dependent losses in insurance

A three dimensional stochastic Model for Claim Reserving

HPOSS: A hierarchical portfolio optimization stacking strategy to reduce the generalization error of ensembles of models

A Bayesian Skew-heavy-tailed modelling for loss reserving

Learning Solutions of Stochastic Optimization Problems with Bayesian Neural Networks

A Semi-Parametric Claims Reserving Model with Monotone Splines

Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction