Abstract:In the past decades, model averaging (MA) has attracted much attention as it has emerged as an alternative tool to the model selection (MS) statistical approach. Hansen [Econometrica 75 (2007) 1175--1189] introduced a Mallows model averaging (MMA) method with model weights selected by minimizing a Mallows' $C_p$ criterion. The main theoretical justification for MMA is an asymptotic optimality (AOP), which states that the risk/loss of the resulting MA estimator is asymptotically equivalent to that of the best but infeasible averaged model. MMA's AOP is proved in the literature by either constraining weights in a special discrete weight set or limiting the number of candidate models. In this work, it is first shown that under these restrictions, however, the optimal risk of MA becomes an unreachable target, and MMA may converge more slowly than MS. In this background, a foundational issue that has not been addressed is: When a suitably large set of candidate models is considered, and the model weights are not harmfully constrained, can the MMA estimator perform asymptotically as well as the optimal convex combination of the candidate models? We answer this question in both nested and non-nested settings. In the nested setting, we provide finite sample inequalities for the risk of MMA and show that without unnatural restrictions on the candidate models, MMA's AOP holds in a general continuous weight set under certain mild conditions. In the non-nested setting, a sufficient condition and a negative result are established for the achievability of the optimal MA risk. Implications on minimax adaptivity are given as well. The results from simulations back up our theoretical findings.

Optimal Model Average Prediction in Orthogonal Kriging Models

Kriging Model Averaging Based on Leave-One-Out Cross-Validation Method

MODEL AVERAGING BASED ON KULLBACK-LEIBLER DISTANCE.

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Model averaging prediction by K -fold cross-validation

Penalized Time-Varying Model Averaging

Optimal Model Averaging for Divergent-Dimensional Poisson Regressions

Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

Jackknife Model Averaging for Additive Expectile Prediction

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Optimal model averaging for divergent-dimensional Poisson regressions with an application on corporate innovation outcomes∗

On optimality of Mallows model averaging

Model averaging prediction for time series models with a diverging number of parameters

Model Averaging with Averaging Covariance Matrix

Post-averaging inference for optimal model averaging estimator in generalized linear models

Model averaging for multivariate multiple regression models

Optimal model averaging based on forward-validation

A Spatiotemporal Regression-Kriging Model for Space-Time Interpolation: a Case Study of Chlorophyll-a Prediction in the Coastal Areas of Zhejiang, China.

A Novel Adaptive Bayesian Model Averaging-Based Multiple Kriging Method for Structural Reliability Analysis

Model averaging in a multiplicative heteroscedastic model

Optimal Designs for Model Averaging in non-nested Models