Abstract:[1] When conducting model averaging for assessing groundwater conceptual model uncertainty, the averaging weights are often evaluated using model selection criteria such as AIC, AICc, BIC, and KIC (Akaike Information Criterion, Corrected Akaike Information Criterion, Bayesian Information Criterion, and Kashyap Information Criterion, respectively). However, this method often leads to an unrealistic situation in which the best model receives overwhelmingly large averaging weight (close to 100%), which cannot be justified by available data and knowledge. It was found in this study that this problem was caused by using the covariance matrix, C-epsilon, of measurement errors for estimating the negative log likelihood function common to all the model selection criteria. This problem can be resolved by using the covariance matrix, Cek, of total errors (including model errors and measurement errors) to account for the correlation between the total errors. An iterative two-stage method was developed in the context of maximum likelihood inverse modeling to iteratively infer the unknown Cek from the residuals during model calibration. The inferred Cek was then used in the evaluation of model selection criteria and model averaging weights. While this method was limited to serial data using time series techniques in this study, it can be extended to spatial data using geostatistical techniques. The method was first evaluated in a synthetic study and then applied to an experimental study, in which alternative surface complexation models were developed to simulate column experiments of uranium reactive transport. It was found that the total errors of the alternative models were temporally correlated due to the model errors. The iterative two-stage method using Cek resolved the problem that the best model receives 100% model averaging weight, and the resulting model averaging weights were supported by the calibration results and physical understanding of the alternative models. Using Cek obtained from the iterative two-stage method also improved predictive performance of the individual models and model averaging in both synthetic and experimental studies.

Introducing AIC model averaging in ecological niche modeling: a single-algorithm multi-model strategy to account for uncertainty in suitability predictions

A brief guide to model selection, multimodel inference and model averaging in behavioural ecology using Akaike’s information criterion

Uninformative Parameters and Model Selection Using Akaike's Information Criterion

ENMeval: An R package for conducting spatially independent evaluations and estimating optimal model complexity for Maxent ecological niche models

Effects of error covariance structure on estimation of model averaging weights and predictive performance

Posterior Averaging Information Criterion

AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons

The cumulative niche approach: A framework to assess the performance of ecological niche model projections

When and when not to use optimal model averaging

A new null model approach to quantify performance and significance for ecological niche models of species distributions

Multimodel Approaches Are Not the Best Way to Understand Multifactorial Systems

Design of ensemble neural network using the Akaike information criterion

Monte Carlo algorithms for model assessment via conflicting summaries

A Conceptual Introduction to Bayesian Model Averaging

Lack of confidence in ABC model choice

Improving ecological niche models by data mining large environmental datasets for surrogate models

Model averaging approaches to data subset selection

Environmental filtering improves ecological niche models across multiple scales

Robust Bayesian model averaging for the analysis of presence–absence data

Assessing the Significance of Model Selection in Ecology

Rethinking receiver operating characteristic analysis applications in ecological niche modeling