Abstract:Advice taking and related research is dominated by deterministic weighting indices, specifically ratio‐of‐differences‐based formulas for investigating informational influence. Their arithmetic is intuitively simple, but they pose several measurement problems and restrict research to a particular paradigmatic approach. As a solution, we propose to specify how strongly peoples' judgments are influenced by externally provided evidence by fitting corresponding mixed‐effects regression models. Our approach explicitly distinguishes between endogenous components, such as updated beliefs, and exogenous components, such as independent initial judgments and advice. Crucially, mixed‐effects regression coefficients of various exogenous sources of information also reflect individual weighting but are based on a conceptually consistent representation of the endogenous judgment process. The formal derivation of the proposed weighting measures is accompanied by a detailed elaboration on their most important technical and statistical subtleties. We use this modeling approach to revisit empirical findings from several paradigms investigating algorithm aversion, sequential collaboration, and advice taking. In summary, we replicate and extend the original finding of algorithm appreciation and initially demonstrate a lack of evidence for both systematic order effects in sequential collaboration as well as differential weighting of multiple pieces of advice. In addition to opening new avenues for innovative research, appropriate modeling of information sampling and utilization has the potential to increase the reproducibility and replicability of behavioral science. Furthermore, the proposed method is relevant beyond advice taking, as mixed‐effects regression weights can also inform research on related cognitive phenomena such as multidimensional belief updating, anchoring effects, hindsight bias, or attitude change.

Analysis of variance and the magnitude of effects: A general approach.

A note on extending Scheffé's modified multiple-comparison procedure to other analysis situations

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

Analysis of variance for repeated measures designs: Two approaches.

Inference for Mixed Models of Anova Type with High-Dimensional Data

Understanding Factorial Designs, Main Effects, and Interaction Effects: Simply Explained with a Worked Example

A-ComVar: A Flexible Extension of Common Variance Design

Beyond step-down analysis: A new test for decomposing the importance of dependent variables in MANOVA.

A brief note on the common (fixed)-effect meta-analysis model

RESEARCH ON ANALYSIS OF VARIANCE AND DATA INTERPRETATION

Understanding and avoiding the "weights of regression": Heterogeneous effects, misspecification, and longstanding solutions

Advances in meta‐analysis: A unifying modelling framework with measurement error correction

Analysis of variance for repeated measures data: A generalized estimating equations approach

The Analysis of Repeated Measures Designs Involving Multiple Dependent Variables

Path Analysis for Binary Random Variables

Performance of Random Effects Model Estimators under Complex Sampling Designs

Mixed‐effects regression weights for advice taking and related phenomena of information sampling and utilization

A simulation study of mediated effect measures

Bayes Factors for Comparison of Two-Way ANOVA Models

An Overview of Modern Machine Learning Methods for Effect Measure Modification Analyses in High-Dimensional Settings

Bayesian Repeated-Measures Analysis of Variance: An Updated Methodology Implemented in JASP