Abstract:In the past decades, model averaging (MA) has attracted much attention as it has emerged as an alternative tool to the model selection (MS) statistical approach. Hansen [Econometrica 75 (2007) 1175--1189] introduced a Mallows model averaging (MMA) method with model weights selected by minimizing a Mallows' $C_p$ criterion. The main theoretical justification for MMA is an asymptotic optimality (AOP), which states that the risk/loss of the resulting MA estimator is asymptotically equivalent to that of the best but infeasible averaged model. MMA's AOP is proved in the literature by either constraining weights in a special discrete weight set or limiting the number of candidate models. In this work, it is first shown that under these restrictions, however, the optimal risk of MA becomes an unreachable target, and MMA may converge more slowly than MS. In this background, a foundational issue that has not been addressed is: When a suitably large set of candidate models is considered, and the model weights are not harmfully constrained, can the MMA estimator perform asymptotically as well as the optimal convex combination of the candidate models? We answer this question in both nested and non-nested settings. In the nested setting, we provide finite sample inequalities for the risk of MMA and show that without unnatural restrictions on the candidate models, MMA's AOP holds in a general continuous weight set under certain mild conditions. In the non-nested setting, a sufficient condition and a negative result are established for the achievability of the optimal MA risk. Implications on minimax adaptivity are given as well. The results from simulations back up our theoretical findings.

On optimality of Mallows model averaging

Model averaging: A shrinkage perspective

Optimal Capital Allocation Based on Weighted-Mean-Variance Principle

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS

Least Squares Model Averaging by Mallows Criterion

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

A Mallows-Type Model Averaging Estimator for the Varying-Coefficient Partially Linear Model

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

A Scalable Frequentist Model Averaging Method

Optimal model averaging for partially linear models with missing response variables and error‐prone covariates

On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Least Squares Model Averaging for Two Non-Nested Linear Models

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

Model Averaging with Averaging Covariance Matrix

Penalized Time-Varying Model Averaging

Model averaging prediction for time series models with a diverging number of parameters

Model averaging in a multiplicative heteroscedastic model

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Spatial Mallows Model Averaging for Geostatistical Models

Model Selection and Model Averaging for Nonlinear Regression Models

Parsimonious Model Averaging With a Diverging Number of Parameters