Abstract:A vast amount of ecological knowledge generated recently has hinged upon the ability of model selection methods to discriminate among various ecological hypotheses. The last decade has seen the rise of Bayesian hierarchical models in ecology. Consequently, popular tools, such as the AIC, become largely inapplicable and other tools are not universally applicable. We focus on a class of competing Bayesian spatially explicit capture recapture (SECR) models and first apply some of the recommended Bayesian model selection tools: (1) Bayes Factor - using (a) Gelfand-Dey (b) harmonic mean methods, (2) DIC, (3) WAIC and (4) the posterior predictive loss function. In all, we evaluate 25 variants of model selection tools in our study. We evaluate these model selection tools from the standpoint of model selection and parameter estimation by contrasting the choice recommended by a tool with a `true' model. In all, we generate 120 simulated data sets using the true model and assess the frequency with which the true model is selected and how well the tool estimates N (population size). We find that when information content is low, no particular tool can be recommended to help realise, simultaneously, both the goals of model selection and parameter estimation. In such scenarios, we recommend that practitioners utilise our application of Bayes Factor for parameter estimation and recommend the posterior predictive loss approach for model selection when information content is low. When both the objectives are taken together, we recommend the use of our applications of Bayes Factor for Bayesian SECR models. Our study reveals that although new model selection tools are emerging (eg: WAIC) in the applied statistics literature, an uncritical absorption of these new tools (i.e. without assessing their efficacies for the problem at hand) into ecological practice may mislead inferences.

On Model Selection with Summary Statistics

Monte Carlo algorithms for model assessment via conflicting summaries

Lack of confidence in ABC model choice

An automatic adaptive method to combine summary statistics in approximate Bayesian computation

An easy-to-use empirical likelihood ABC method

Model Misspecification in ABC: Consequences and Diagnostics

Why approximate Bayesian computational (ABC) methods cannot handle model choice problems

Summary Statistics in Approximate Bayesian Computation

A Parzen-based distance between probability measures as an alternative of summary statistics in Approximate Bayesian Computation

Stratified sampling and bootstrapping for approximate Bayesian computation

Asymptotics of approximate Bayesian computation when summary statistics converge at heterogeneous rates

Simulation‐based inference for reliability model selection and parameter calibration: An application to fatigue

An Approximate Likelihood Perspective on ABC Methods

Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems

Reconciling the Bayes Factor and Likelihood Ratio for Two Non-Nested Model Selection Problems

ABC-CDE: Towards Approximate Bayesian Computation with Complex High-Dimensional Data and Limited Simulations

Towards Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach

Approximate Bayesian Computation via Classification

Bayesian Model Selection for a Class of Spatially-Explicit Capture Recapture Models

Comparison of Bayesian predictive methods for model selection

Tutorial on ABC rejection and ABC SMC for parameter estimation and model selection