Abstract:Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

Entropy-Based Model Averaging Estimation of Nonparametric Models

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Penalized Time-Varying Model Averaging

Model averaging in a multiplicative heteroscedastic model

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

Optimal Model Averaging for Divergent-Dimensional Poisson Regressions

A Scalable Frequentist Model Averaging Method

A Model Averaging Approach for the Ordered Probit and Nested Logit Models with Applications

Model Averaging with Averaging Covariance Matrix

Jackknife Model Averaging for Additive Expectile Prediction

When and when not to use optimal model averaging

Model Averaging Based on Generalized Method of Moments

Model averaging approaches to data subset selection

Parsimonious Model Averaging With a Diverging Number of Parameters

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Model-averaging-based semiparametric modeling for conditional quantile prediction

Model averaging for right censored data with measurement error

Model Averaging for Estimating Treatment Effects With Binary Responses

Stability and L2-penalty in Model Averaging

Model Averaging and Its Use in Economics