Abstract:Whether deterministic or stochastic, models can be viewed as functions designed to approximate a specific quantity of interest. We propose a data-driven framework that aggregates predictions from diverse models into a single, more accurate output. This aggregation approach exploits each model's strengths to enhance overall accuracy. It is non-intrusive - treating models as black-box functions - model-agnostic, requires minimal assumptions, and can combine outputs from a wide range of models, including those from machine learning and numerical solvers. We argue that the aggregation process should be point-wise linear and propose two methods to find an optimal aggregate: Minimal Error Aggregation (MEA), which minimizes the aggregate's prediction error, and Minimal Variance Aggregation (MVA), which minimizes its variance. While MEA is inherently more accurate when correlations between models and the target quantity are perfectly known, Minimal Empirical Variance Aggregation (MEVA), an empirical version of MVA - consistently outperforms Minimal Empirical Error Aggregation (MEEA), the empirical counterpart of MEA, when these correlations must be estimated from data. The key difference is that MEVA constructs an aggregate by estimating model errors, while MEEA treats the models as features for direct interpolation of the quantity of interest. This makes MEEA more susceptible to overfitting and poor generalization, where the aggregate may underperform individual models during testing. We demonstrate the versatility and effectiveness of our framework in various applications, such as data science and partial differential equations, showing how it successfully integrates traditional solvers with machine learning models to improve both robustness and accuracy.

Empirical risk minimization is consistent with the mean absolute percentage error

Using the Mean Absolute Percentage Error for Regression Models

Mean Absolute Percentage Error for regression models

A better measure of relative prediction accuracy for model selection and model estimation

Empirical Risk Minimization with Relative Entropy Regularization

Error Density-dependent Empirical Risk Minimization

Empirical MSE Minimization to Estimate a Scalar Parameter

Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not

Nonconcave Penalized M-estimation for the Least Absolute Relative Errors Model

Risk-Aware MMSE Estimation

A simulation study of the model evaluation criterion mmre

Model aggregation: minimizing empirical variance outperforms minimizing empirical error

Ermma: Expected Risk Minimization for Matrix Approximation-Based Recommender Systems

A New Interpretation on the MMSE As a Robust MEE Criterion

Consistency analysis of an empirical minimum error entropy algorithm

Self-Averaging Property of Minimal Investment Risk of Mean-Variance Model

Least Absolute Relative Error Estimation

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

Optimal estimator for uncertainty-based measurement quality control

Insights into the Robustness of Minimum Error Entropy Estimation