Abstract:Frequentist model averaging is an effective technique to handle model uncertainty. However, calculation of the weights for averaging is extremely difficult, if not impossible, even when the dimension of the predictor vector, p, is moderate, because we may have 2(p) candidate models. The exponential size of the candidate model set makes it difficult to estimate all candidate models, and brings additional numeric errors when calculating the weights. This article proposes a scalable frequentist model averaging method, which is statistically and computationally efficient, to overcome this problem by transforming the original model using the singular value decomposition. The method enables us to find the optimal weights by considering at most p candidate models. We prove that the minimum loss of the scalable model averaging estimator is asymptotically equal to that of the traditional model averaging estimator. We apply the Mallows and Jackknife criteria to the scalable model averaging estimator and prove that they are asymptotically optimal estimators. We further extend the method to the high-dimensional case (i.e., p >= n). Numerical studies illustrate the superiority of the proposed method in terms of both statistical efficiency and computational cost.

Averaging Estimators for Discrete Choice by M-fold Cross-Validation

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Model Averaging for Estimating Treatment Effects With Binary Responses

Jackknife Model Averaging for Additive Expectile Prediction

Model averaging prediction by K -fold cross-validation

A Model Averaging Approach for the Ordered Probit and Nested Logit Models with Applications

Model Averaging Based on Leave-Subject-out Cross-Validation

Model Averaging Based on Generalized Method of Moments

Unified Optimal Model Averaging with a General Loss Function Based on Cross-Validation

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Jackknife Model Averaging for Composite Quantile Regression

Model Averaging for Support Vector Machine by J − fold Cross-Validation *

Model Averaging for Support Vector Classifier by Cross-Validation

Jackknife model averaging for linear regression models with missing responses

A Scalable Frequentist Model Averaging Method

Cross-Validation Model Averaging for Generalized Functional Linear Model

Kriging Model Averaging Based on Leave-One-Out Cross-Validation Method

A general averaging method for count data with overdispersion and/or excess zeros in biomedicine

Estimating Average Treatment Effect by Model Averaging

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Penalized Time-Varying Model Averaging