Abstract:Research on best practices in theory assessment highlights that testing theories is challenging because they inherit a new set of assumptions as soon as they are linked to a specific methodology. In this article, we integrate and build on this work by demonstrating the breadth of these challenges. We show that tracking auxiliary assumptions is difficult because they are made at different stages of theory testing and at multiple levels of a theory. We focus on these issues in a reanalysis of a seminal study and its replications, both of which use a simple working-memory paradigm and a mainstream computational modeling approach. These studies provide the main evidence for "all-or-none" recognition models of visual working memory and are still used as the basis for how to measure performance in popular visual working-memory tasks. In our reanalysis, we find that core practical auxiliary assumptions were unchecked and violated; the original model comparison metrics and data were not diagnostic in several experiments. Furthermore, we find that models were not matched on "theory general" auxiliary assumptions, meaning that the set of tested models was restricted, and not matched in theoretical scope. After testing these auxiliary assumptions and identifying diagnostic testing conditions, we find evidence for the opposite conclusion. That is, continuous resource models outperform all-or-none models. Together, our work demonstrates why tracking and testing auxiliary assumptions remains a fundamental challenge, even in prominent studies led by careful, computationally minded researchers. Our work also serves as a conceptual guide on how to identify and test the gamut of auxiliary assumptions in theory assessment, and we discuss these ideas in the context of contemporary approaches to scientific discovery.

A working memory dependent dual process model of the testing effect

Is Working Memory Necessary for Both Selection and Calculation in Level 1 Visual Perspective Taking? Evidence from Children and Adults

Individual Differences in Working Memory Capacity Are Reflected in Different ERP and EEG Patterns to Task Difficulty

Dual-task Costs in Memory Recall Precision Reflect Shared Representational Space

Do working memory capacity and test anxiety modulate the beneficial effects of testing on new learning?

The impact of working memory testing on long-term associative memory

Testing the dual-memory framework: individual differences in the magnitude of the retrieval practice effect and fluid intelligence

The dual-process perspective and the benefits of retrieval practice in younger and older adults

Neural correlates of retrieval-based memory enhancement: An fMRI study of the testing effect

Long-Term Effects of Working Memory Retrieval From Prioritized and Deprioritized States

Zooming in on what counts as core and auxiliary: A case study on recognition models of visual working memory

A new aspect of cognitive selectivity: Working memory reselection for attended information

Retrieval practice effect and individual differences: current status and future directions

Working Memory Guides Action Valuation in Model-based Decision-making Strategy.

Modeling working memory: a computational implementation of the Time-Based Resource-Sharing theory

EXPRESS: From task-general towards task-specific cognitive operations in a few minutes? Working memory performance as an adaptive process

Significant Retest Effects in Spatial Working Memory Task

The effect of working memory maintenance on long-term memory

Working Memory Resources Depletion Makes Delayed Testing Beneficial

The Testing Effect in University Teaching: Using Multiple-Choice Testing to Promote Retention of Highly Retrievable Information

Behavioral and EEG Effects of Working Memory Updating Training in Children with Learning Disabilities