Abstract:Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

Model Averaging Based on James–Stein Estimators

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Penalized Time-Varying Model Averaging

Model averaging in a multiplicative heteroscedastic model

Model Averaging with Averaging Covariance Matrix

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

A Scalable Frequentist Model Averaging Method

A Class of Model Averaging Estimators

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Model Averaging Based on Generalized Method of Moments

Model averaging estimation for generalized partially linear varying-coefficient models

Jackknife Model Averaging for Additive Expectile Prediction

Optimal Model Averaging for Divergent-Dimensional Poisson Regressions

A Model Averaging Approach for the Ordered Probit and Nested Logit Models with Applications

Jackknife Model Averaging for Quantile Single-Index Coefficient Model

Optimal Model Averaging Estimator for Multinomial Logit Models

Model Averaging Based on Leave-Subject-out Cross-Validation

Stability and L2-penalty in Model Averaging

Model Averaging for Semiparametric Varying Coefficient Quantile Regression Models

A Mallows-type Model Averaging Estimator for De-Noise Linear Models

Model Averaging Estimation for Conditional Volatility Models