Abstract:Surrogate models are frequently used to replace costly engineering simulations. A single surrogate is frequently chosen based on previous experience or by fitting multiple surrogates and selecting one based on mean cross-validation errors. A novel stacking strategy will be presented in this paper. This new strategy results from reinterpreting the model selection process based on the generalization error. For the first time, this problem is proposed to be translated into a well-studied financial problem: portfolio management and optimization. In short, it is demonstrated that the individual residues calculated by leave-one-out procedures are samples from a given random variable ϵi, whose second non-central moment is the i-th model's generalization error. Thus, a stacking methodology based solely on evaluating the behavior of the linear combination of the random variables ϵi is proposed. At first, several surrogate models are calibrated. The Directed Bubble Hierarchical Tree (DBHT) clustering algorithm is then used to determine which models are worth stacking. The stacking weights can be calculated using any financial approach to the portfolio optimization problem. This alternative understanding of the problem enables practitioners to use established financial methodologies to calculate the models' weights, significantly improving the ensemble of models' out-of-sample performance. A study case is carried out to demonstrate the applicability of the new methodology. Overall, a total of 124 models were trained using a specific dataset: 40 Machine Learning models and 84 Polynomial Chaos Expansion models (which considered 3 types of base random variables, 7 least square algorithms for fitting the up to fourth order expansion's coefficients). Among those, 99 models could be fitted without convergence and other numerical issues. The DBHT algorithm with Pearson correlation distance and generalization error similarity was able to select a subgroup of 23 models from the 99 fitted ones, implying a reduction of about 77% in the total number of models, representing a good filtering scheme which still preserves diversity. Finally, it has been demonstrated that the weights obtained by building a Hierarchical Risk Parity (HPR) portfolio perform better for various input random variables, indicating better out-of-sample performance. In this way, an economic stacking strategy has demonstrated its worth in improving the out-of-sample capabilities of stacked models, which illustrates how the new understanding of model stacking methodologies may be useful.

StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics

MetaStackVis: Visually-Assisted Performance Evaluation of Metamodels

A Generalized Stacking for Implementing Ensembles of Gradient Boosting Machines

Enhancing binary classification: A new stacking method via leveraging computational geometry

Stacking and stability

Issues in Stacked Generalization

Stacking Ensemble Technique for Classifying Breast Cancer

Cost-Sensitive Stacking: an Empirical Evaluation

Feature-Weighted Linear Stacking

HPOSS: A hierarchical portfolio optimization stacking strategy to reduce the generalization error of ensembles of models

View selection in multi-view stacking: choosing the meta-learner

Systematic Ensemble Learning for Regression

Effective training-time stacking for ensembling of deep neural networks

Stacked Generalization: when Does It Work?

Exploration of the Stacking Ensemble Machine Learning Algorithm for Cheating Detection in Large-Scale Assessment

ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking

A Stack-Centric Processing Model for Iterative Processing

Evaluating StackingC and ensemble models for enhanced lithological classification in geological mapping

pystacked: Stacking generalization and machine learning in Stata

SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks

Dynamic Stacked Generalization for Node Classification on Networks