Abstract:In the past decades, model averaging (MA) has attracted much attention as it has emerged as an alternative tool to the model selection (MS) statistical approach. Hansen [Econometrica 75 (2007) 1175--1189] introduced a Mallows model averaging (MMA) method with model weights selected by minimizing a Mallows' $C_p$ criterion. The main theoretical justification for MMA is an asymptotic optimality (AOP), which states that the risk/loss of the resulting MA estimator is asymptotically equivalent to that of the best but infeasible averaged model. MMA's AOP is proved in the literature by either constraining weights in a special discrete weight set or limiting the number of candidate models. In this work, it is first shown that under these restrictions, however, the optimal risk of MA becomes an unreachable target, and MMA may converge more slowly than MS. In this background, a foundational issue that has not been addressed is: When a suitably large set of candidate models is considered, and the model weights are not harmfully constrained, can the MMA estimator perform asymptotically as well as the optimal convex combination of the candidate models? We answer this question in both nested and non-nested settings. In the nested setting, we provide finite sample inequalities for the risk of MMA and show that without unnatural restrictions on the candidate models, MMA's AOP holds in a general continuous weight set under certain mild conditions. In the non-nested setting, a sufficient condition and a negative result are established for the achievability of the optimal MA risk. Implications on minimax adaptivity are given as well. The results from simulations back up our theoretical findings.

Least Squares Model Averaging for Two Non-Nested Linear Models

Least Squares Model Averaging by Mallows Criterion

Model Selection and Model Averaging for Nonlinear Regression Models

A Mallows-type Model Averaging Estimator for De-Noise Linear Models

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Model Averaging with Averaging Covariance Matrix

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS

Least Squares Model Averaging Based on Generalized Cross Validation

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Penalized Time-Varying Model Averaging

Spatial Mallows Model Averaging for Geostatistical Models

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

Optimal Model Averaging of Varying Coefficient Models

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

On optimality of Mallows model averaging

Model averaging in a multiplicative heteroscedastic model

Optimal Model Averaging for Semiparametric Partially Linear Models with Measurement Errors

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Model averaging for right censored data with measurement error

Model Averaging for Nonlinear Regression Models

Model averaging estimation for generalized partially linear varying-coefficient models