Abstract:Frequentist model averaging is an effective technique to handle model uncertainty. However, calculation of the weights for averaging is extremely difficult, if not impossible, even when the dimension of the predictor vector, p, is moderate, because we may have 2(p) candidate models. The exponential size of the candidate model set makes it difficult to estimate all candidate models, and brings additional numeric errors when calculating the weights. This article proposes a scalable frequentist model averaging method, which is statistically and computationally efficient, to overcome this problem by transforming the original model using the singular value decomposition. The method enables us to find the optimal weights by considering at most p candidate models. We prove that the minimum loss of the scalable model averaging estimator is asymptotically equal to that of the traditional model averaging estimator. We apply the Mallows and Jackknife criteria to the scalable model averaging estimator and prove that they are asymptotically optimal estimators. We further extend the method to the high-dimensional case (i.e., p >= n). Numerical studies illustrate the superiority of the proposed method in terms of both statistical efficiency and computational cost.

Post Averaging Inference under Heteroskedasticity

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Model averaging in a multiplicative heteroscedastic model

Post-averaging inference for optimal model averaging estimator in generalized linear models

Penalized Time-Varying Model Averaging

Jackknife Model Averaging for Additive Expectile Prediction

Parsimonious Model Averaging With a Diverging Number of Parameters

Model Averaging with Averaging Covariance Matrix

Model averaging estimation for generalized partially linear varying-coefficient models

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

A Model Averaging Approach for the Ordered Probit and Nested Logit Models with Applications

A Scalable Frequentist Model Averaging Method

Model Averaging for M-estimation

Estimating Conditional Average Treatment Effects with Heteroscedasticity by Model Averaging and Matching

Model Averaging and Its Use in Economics

Optimal Model Averaging for Semiparametric Partially Linear Models with Censored Data

Optimal Model Averaging for Divergent-Dimensional Poisson Regressions

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Partial Linear Model Averaging Prediction for Longitudinal Data

Estimating Average Treatment Effect by Model Averaging

Model Selection and Model Averaging for Semiparametric Partially Linear Models with Missing Data