Abstract:Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

Penalized Time-Varying Model Averaging

Time-varying model averaging

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Model averaging prediction for time series models with a diverging number of parameters

A Scalable Frequentist Model Averaging Method

Optimal model averaging based on forward-validation

Parsimonious Model Averaging With a Diverging Number of Parameters

Partial Linear Model Averaging Prediction for Longitudinal Data

Jackknife Model Averaging for Additive Expectile Prediction

Penalised inference for autoregressive moving average models with time-dependent predictors

Stability and L2-penalty in Model Averaging

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Post-averaging inference for optimal model averaging estimator in generalized linear models

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Model Switching and Model Averaging in Time-Varying Parameter Regression Models

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Model Averaging for Accelerated Failure Time Models with Missing Censoring Indicators

Entropy-Based Model Averaging Estimation of Nonparametric Models

Optimal model averaging for partially linear models with missing response variables and error‐prone covariates

Model averaging prediction by K -fold cross-validation

Model Averaging for Generalized Linear Model with Covariates that are Missing completely at Random