Model-averaged Bayesian t tests

Maximilian Maier,František Bartoš,Daniel S. Quintana,Fabian Dablander,Don van den Bergh,Maarten Marsman,Alexander Ly,Eric-Jan Wagenmakers

DOI: https://doi.org/10.3758/s13423-024-02590-5

2024-11-09

Psychonomic Bulletin & Review

Abstract:One of the most common statistical analyses in experimental psychology concerns the comparison of two means using the frequentist t  test. However, frequentist t  tests do not quantify evidence and require various assumption tests. Recently, popularized Bayesian t  tests do quantify evidence, but these were developed for scenarios where the two populations are assumed to have the same variance. As an alternative to both methods, we outline a comprehensive t  test framework based on Bayesian model averaging. This new t  test framework simultaneously takes into account models that assume equal and unequal variances, and models that use t -likelihoods to improve robustness to outliers. The resulting inference is based on a weighted average across the entire model ensemble, with higher weights assigned to models that predicted the observed data well. This new t  test framework provides an integrated approach to assumption checks and inference by applying a series of pertinent models to the data simultaneously rather than sequentially. The integrated Bayesian model-averaged t  tests achieve robustness without having to commit to a single model following a series of assumption checks. To facilitate practical applications, we provide user-friendly implementations in JASP and via the package in . A tutorial video is available at https://www.youtube.com/watch?v=EcuzGTIcorQ

psychology, experimental, mathematical