Abstract:Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

A Class of Model Averaging Estimators

Model Averaging Based on James–Stein Estimators

Model Averaging Based on Generalized Method of Moments

Model Averaging with Averaging Covariance Matrix

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Model averaging in a multiplicative heteroscedastic model

Jackknife Model Averaging for Additive Expectile Prediction

Penalized Time-Varying Model Averaging

On the dominance of Mallows model averaging estimator over ordinary least squares estimator

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Model Averaging for Generalized Linear Model with Covariates that are Missing completely at Random

Estimating Average Treatment Effect by Model Averaging

Least Squares Model Averaging Based on Generalized Cross Validation

Post Averaging Inference under Heteroskedasticity

Model Averaging Based on Leave-Subject-out Cross-Validation

Post-averaging inference for optimal model averaging estimator in generalized linear models

Stability and L2-penalty in Model Averaging

Unified Optimal Model Averaging with a General Loss Function Based on Cross-Validation

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Model Averaging Estimation for Conditional Volatility Models