Abstract:As use of Akaike's Information Criterion (AIC) for model selection has become increasingly common, so has a mistake involving interpretation of models that are within 2 AIC units (AIC 2) of the top-supported model. Such models are <2 AIC units because the penalty for one additional parameter is 2 AIC units, but model deviance is not reduced by an amount sufficient to overcome the 2-unit penalty and, hence, the additional parameter provides no net reduction in AIC. Simply put, the uninformative parameter does not explain enough variation to justify its inclusion in the model and it should not be interpreted as having any ecological effect. Models with uninformative parameters are frequently presented as being competitive in the Journal of Wildlife Management, including 72 of all AIC-based papers in 2008, and authors and readers need to be more aware of this problem and take appropriate steps to eliminate misinterpretation. I reviewed 5 potential solutions to this problem: 1) report all models but ignore or dismiss those with uninformative parameters, 2) use model averaging to ameliorate the effect of uninformative parameters, 3) use 95 confidence intervals to identify uninformative parameters, 4) perform all-possible subsets regression and use weight-of-evidence approaches to discriminate useful from uninformative parameters, or 5) adopt a methodological approach that allows models containing uninformative parameters to be culled from reported model sets. The first approach is preferable for small sets of a priori models, whereas the last 2 approaches should be used for large model sets or exploratory modeling.

Extending AIC to best subset regression

Variable Selection with Stepwise and Best Subset Approaches.

A simple application of FIC to model selection

Minimization of Akaike's information criterion in linear regression analysis via mixed integer nonlinear program

A Theoretical Investigation of Several Model Selection Criteria for Dimensionality Reduction

AIC under the framework of least squares estimation

Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).

Factor Analysis and AIC

Rebuilding Factorized Information Criterion: Asymptotically Accurate Marginal Likelihood

A brief guide to model selection, multimodel inference and model averaging in behavioural ecology using Akaike’s information criterion

On the Equivalence of Factorized Information Criterion Regularization and the Chinese Restaurant Process Prior

Consistent Model Selection Procedure for Random Coefficient INAR Models

Posterior Averaging Information Criterion

Variable Selection in Data Envelopment Analysis Via Akaike’s Information Criteria

An Information Analysis on Modeling Interaction Effects in Logistic Regression

Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso

Uninformative Parameters and Model Selection Using Akaike's Information Criterion

Theoretical Analysis and Comparison of Several Criteria on Linear Model Dimension Reduction

A Consistent and Scalable Algorithm for Best Subset Selection in Single Index Models

A Unified Approach to Focused Information Criterion and Plug-In Averaging Method

Extended Bayesian Information Criterion in the Cox Model with a High-Dimensional Feature Space