Abstract:Surrogate models are frequently used to replace costly engineering simulations. A single surrogate is frequently chosen based on previous experience or by fitting multiple surrogates and selecting one based on mean cross-validation errors. A novel stacking strategy will be presented in this paper. This new strategy results from reinterpreting the model selection process based on the generalization error. For the first time, this problem is proposed to be translated into a well-studied financial problem: portfolio management and optimization. In short, it is demonstrated that the individual residues calculated by leave-one-out procedures are samples from a given random variable ϵi, whose second non-central moment is the i-th model's generalization error. Thus, a stacking methodology based solely on evaluating the behavior of the linear combination of the random variables ϵi is proposed. At first, several surrogate models are calibrated. The Directed Bubble Hierarchical Tree (DBHT) clustering algorithm is then used to determine which models are worth stacking. The stacking weights can be calculated using any financial approach to the portfolio optimization problem. This alternative understanding of the problem enables practitioners to use established financial methodologies to calculate the models' weights, significantly improving the ensemble of models' out-of-sample performance. A study case is carried out to demonstrate the applicability of the new methodology. Overall, a total of 124 models were trained using a specific dataset: 40 Machine Learning models and 84 Polynomial Chaos Expansion models (which considered 3 types of base random variables, 7 least square algorithms for fitting the up to fourth order expansion's coefficients). Among those, 99 models could be fitted without convergence and other numerical issues. The DBHT algorithm with Pearson correlation distance and generalization error similarity was able to select a subgroup of 23 models from the 99 fitted ones, implying a reduction of about 77% in the total number of models, representing a good filtering scheme which still preserves diversity. Finally, it has been demonstrated that the weights obtained by building a Hierarchical Risk Parity (HPR) portfolio perform better for various input random variables, indicating better out-of-sample performance. In this way, an economic stacking strategy has demonstrated its worth in improving the out-of-sample capabilities of stacked models, which illustrates how the new understanding of model stacking methodologies may be useful.

Schur Complementary Allocation: A Unification of Hierarchical Risk Parity and Minimum Variance Portfolios

Diversified reward-risk parity in portfolio construction

Constrained Risk Budgeting Portfolios: Theory, Algorithms, Applications & Puzzles

Derivatives-based portfolio decisions. An expected utility insight

A Unified Formula of the Optimal Portfolio for Piecewise HARA Utilities

Counter-monotonic risk allocations and distortion risk measures

Portfolio risk allocation through Shapley value

HPOSS: A hierarchical portfolio optimization stacking strategy to reduce the generalization error of ensembles of models

UNIVERSAL RISK BUDGETING

A Unified Framework for Fast Large-Scale Portfolio Optimization

Introduction to Risk Parity and Budgeting

Portfolio Optimization Rules beyond the Mean-Variance Approach

Robust mean-risk portfolio optimization using machine learning-based trade-off parameter

Risk Budgeting Portfolios: Existence and Computation

A General Framework for Portfolio Theory. Part I: theory and various models

Risk parity portfolio optimization under heavy‐tailed returns and dynamic correlations

On Capital Allocation under Information Constraints

Risk and Utility in Portfolio Optimization

Schrödinger Risk Diversification Portfolio

A Fuzzy Entropy Approach for Portfolio Selection

Portfolio Optimization with Allocation Constraints and Stochastic Factor Market Dynamics