Abstract:Perceptual confidence results from a metacognitive process which evaluates how likely our percepts are to be correct. Many competing models of perceptual metacognition enjoy strong empirical support. Arbitrating these models traditionally proceeds via researchers conducting experiments and then fitting several models to the data collected. However, such a process often includes conditions or paradigms that may not best arbitrate competing models: Many models make similar predictions under typical experimental conditions. Consequently, many experiments are needed, collectively (sub-optimally) sampling the space of conditions to compare models. Here, instead, we introduce a variant of optimal experimental design which we call a computational-rationality approach to generative models of cognition, using perceptual metacognition as a case study. Instead of designing experiments and post-hoc specifying models, we began with comprehensive model comparison among four competing generative models for perceptual metacognition, drawn from literature. By simulating a simple experiment under each model, we identified conditions where these models made maximally diverging predictions for confidence. We then presented these conditions to human observers, and compared the models' capacity to predict choices and confidence. Results revealed two surprising findings: (1) two models previously reported to differently predict confidence to different degrees, with one predicting better than the other, appeared to predict confidence in a direction opposite to previous findings; and (2) two other models previously reported to equivalently predict confidence showed stark differences in the conditions tested here. Although preliminary with regards to which model is actually 'correct' for perceptual metacognition, our findings reveal the promise of this computational-rationality approach to maximizing experimental utility in model arbitration while minimizing the number of experiments necessary to reveal the winning model, both for perceptual metacognition and in other domains.

Assessing metacognitive knowledge in subjective decisions: The knowledge of weights paradigm

Metacognitive Information Theory

Putting post-decision wagering to the test: a measure of self-perceived knowledge in basic sciences?

Parental rights or parental wrongs: Parents’ metacognitive knowledge of the factors that influence their school choice decisions

Ant Colony Optimization and Local Weighted Structural Equation Modeling. A Tutorial on Novel Item and Person Sampling Procedures for Personality Research

Mixed‐effects regression weights for advice taking and related phenomena of information sampling and utilization

A Preliminary Study of Human Decision-Making, Risk Attitude, and Social Preference on Knowledge Management

An ecological assessment of decision-making under risk and ambiguity through the virtual serious game Kalliste Decision Task

The neural representation of metacognition in preferential decision‐making

Cognitive processes underlying lottery and sports gambling decisions: The role of stated probabilities and background knowledge

An Evaluation of Daily Perceived Stress and Impulsive Decision Making: A Pilot Study

Knowing whom to learn from: individual differences in metacognition and weighting of social information

The assessment of affective decision‐making: Exploring alternative scoring methods for the Balloon Analog Risk Task and Columbia Card Task

Relying on External Information Sources When Answering Knowledge Questions in Web Surveys

Ecological validity of common behavioral decision making tasks: evidence across two samples

Working Memory Workload When Making Complex Decisions: A Behavioral and EEG Study

Quantitative vs. Qualitative Outcomes: A Longitudinal Study of Risk and Ambiguity in Monetary and Medical Decision-Making

Simple additive weighting—A metamodel for multiple criteria decision analysis methods

Toward 'Computational-Rationality' Approaches to Arbitrating Models of Cognition: A Case Study Using Perceptual Metacognition

Modeling Decision-Making Under Uncertainty with Qualitative Outcomes

Self-Regulated Learning: The Interactive Influence of Metacognitive Awareness and Goal-Setting