Abstract:Advice taking and related research is dominated by deterministic weighting indices, specifically ratio‐of‐differences‐based formulas for investigating informational influence. Their arithmetic is intuitively simple, but they pose several measurement problems and restrict research to a particular paradigmatic approach. As a solution, we propose to specify how strongly peoples' judgments are influenced by externally provided evidence by fitting corresponding mixed‐effects regression models. Our approach explicitly distinguishes between endogenous components, such as updated beliefs, and exogenous components, such as independent initial judgments and advice. Crucially, mixed‐effects regression coefficients of various exogenous sources of information also reflect individual weighting but are based on a conceptually consistent representation of the endogenous judgment process. The formal derivation of the proposed weighting measures is accompanied by a detailed elaboration on their most important technical and statistical subtleties. We use this modeling approach to revisit empirical findings from several paradigms investigating algorithm aversion, sequential collaboration, and advice taking. In summary, we replicate and extend the original finding of algorithm appreciation and initially demonstrate a lack of evidence for both systematic order effects in sequential collaboration as well as differential weighting of multiple pieces of advice. In addition to opening new avenues for innovative research, appropriate modeling of information sampling and utilization has the potential to increase the reproducibility and replicability of behavioral science. Furthermore, the proposed method is relevant beyond advice taking, as mixed‐effects regression weights can also inform research on related cognitive phenomena such as multidimensional belief updating, anchoring effects, hindsight bias, or attitude change.

Diagnostic Tests for the Necessity of Weight in Regression with Survey Data

Understanding and avoiding the "weights of regression": Heterogeneous effects, misspecification, and longstanding solutions

To Weight or Not to Weight? Balancing Influence of Initial Items in Adaptive Testing

New multivariate tests for assessing covariate balance in matched observational studies

Consistent Specification Testing Via Nonparametric Series Regression

Variance-based sensitivity analysis for weighting estimators result in more informative bounds

A Shrinkage Method for Testing the Hardy–Weinberg Equilibrium in Case‐Control Studies

On the implied weights of linear regression for causal inference

Evaluating Binary Outcome Classifiers Estimated from Survey Data

A Comparison of Approaches to Control for Confounding Factors by Regression Models

A modified regression method to test publication bias in meta-analyses with binary outcomes

Mixed‐effects regression weights for advice taking and related phenomena of information sampling and utilization

A difference-based method for testing no effect in nonparametric regression

Optimization of Survey Weights under a Large Number of Conflicting Constraints

Wald tests for variance-adjusted equivalence assessment with normal endpoints

Testing two variances for superiority/non-inferiority and equivalence: Using the exhaustion algorithm for sample size allocation with cost

Optimization for Calibration of Survey Weights under a Large Number of Conflicting Constraints

Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?

Minimal dispersion approximately balancing weights: asymptotic properties and practical considerations

Weighted residual empirical processes, martingale transformations and model checking for regressions

To weight or not to weight? The effect of selection bias in 3 large electronic health record-linked biobanks and recommendations for practice