Abstract:Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

Unified Optimal Model Averaging with a General Loss Function Based on Cross-Validation

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Optimal Capital Allocation Based on Weighted-Mean-Variance Principle

Penalized Time-Varying Model Averaging

Optimal model averaging for partially linear models with missing response variables and error‐prone covariates

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Ultra-High Dimensional Model Averaging for Multi-Categorical Response

Model averaging prediction by K -fold cross-validation

Model Averaging for Generalized Linear Model with Covariates that are Missing completely at Random

A Scalable Frequentist Model Averaging Method

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Cross-Validation Model Averaging for Generalized Functional Linear Model

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Post-averaging inference for optimal model averaging estimator in generalized linear models

Stability and L2-penalty in Model Averaging

Parsimonious Model Averaging With a Diverging Number of Parameters

Partial Linear Model Averaging Prediction for Longitudinal Data

Optimal model averaging based on forward-validation

Model Averaging for Generalized Linear Models in Fragmentary Data Prediction

Model Averaging for Estimating Treatment Effects With Binary Responses

Jackknife model averaging for linear regression models with missing responses