Abstract:Background. Online, algorithmically driven prognostic tools are increasingly important in medical decision making. Institutions developing such tools need to be able to communicate the precision and accuracy of the information in a trustworthy manner, and so many attempt to communicate uncertainties but also use institutional logos to underscore their trustworthiness. Bringing together theories on trust, uncertainty, and psychological distance in a novel way, we tested whether and how the communication of uncertainty and the presence of institutional logos together affected trust in medical information, the prognostic tool itself, and treatment decisions. Methods. A pilot and 2 online experiments in which UK (experiment 1) and worldwide (experiment 2) participants (Ntotal = 4,724) were randomized to 1 of 12 arms in a 3 (uncertainty cue) × 4 (institutional cue) between-subjects design. The stimulus was based on an existing medical prognostic tool. Results. Institutional trust was consistently associated with trust in the prognostic tool itself, while uncertainty information had no consistent effect. Institutional trust predicted the amount of weight participants reported placing on institutional endorsements in their decision making and the likelihood of switching from passive to active treatment in a hypothetical scenario. There was also a significant effect of psychological distance to (perceived hypotheticality of) the scenario. Conclusions/Implications. These results underline the importance of institutions demonstrating trustworthiness and building trust with their users. They also suggest that users tend to be insensitive to communications of uncertainty and that communicators may need to be highly explicit when attempting to warn of low precision or quality of evidence. The effect of the perceived hypotheticality of the scenario underscores the importance of realistic decision-making scenarios for studies and the role of familiarity with the decision dilemma generally. Highlights: In a world where information for medical decision making is increasingly going to be provided through digital, online tools, institutions providing such tools need guidance on how best to communicate about their trustworthiness and precision.We find that people are fairly insensitive to cues designed to communicate uncertainty around the outputs of such tools. Even putting "ATTENTION" in bold font or explicitly pointing out the weaknesses in the data did not appear to affect people's decision making using the tool's outputs. Institutions should take note, and further work is required to determine how best to communicate uncertainty in a way that elicits appropriate caution in lay users.People were much more sensitive to institutional logos associated with the outputs. Generalized institutional trust (rather than trust in the specific institution whose logo was shown) was associated with how trustworthy, accurate, and reliable the tool, its algorithm, and the numbers it produced were perceived to be. This underscores the role of societal trust in institutions at large.Finally, as a note to researchers, we found a significant effect of how hypothetical or believable participants felt the experimental scenario was. This is a variable that seems rarely controlled for in studies and yet played as much of a role as some of our variables of interest, so we suggest that it is measured in future experiments.

Delta Inflation, Optimism Bias, and Uncertainty in Clinical Trials

Why There Are So Many Contradicted or Exaggerated Findings in Highly-Cited Clinical Research?

Towards More Flexible False Positive Control in Phase III Randomized Clinical Trials

Asymptotic properties of doubly adaptive biased coin designs for multitreatment clinical trials

Negative Spillover: A Potential Source of Bias in Pragmatic Clinical Trials

What Affects Perceived Trustworthiness of Online Medical Information and Subsequent Treatment Decision Making? Randomized Trials on the Role of Uncertainty and Institutional Cues

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials

Statistical Decision Properties of Imprecise Trials Assessing COVID-19 Drugs

Post Hoc Bias in Treatment Decisions

Disproportional inflation of clinical trial costs: why we should care, and what we should do about it

A conservative approach to leveraging external evidence for effective clinical trial design

Estimands in clinical trials of complex disease processes

Conditional and Unconditional Treatment Effects in Randomized Clinical Trials: Estimands, Estimation, and Interpretation

Some Statistical Issues of Confirming Efficacy in Clinical Trials

Estimation of ascertainment bias and its effect on power in clinical trials with time-to-event outcomes

Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors

A behavioral economics perspective on the COVID-19 vaccine amid public mistrust

How Much Should We Trust Staggered Difference-In-Differences Estimates?

Uncertainty Quantification and Interpretability for Clinical Trial Approval Prediction

Radical Thinking: Scientific Rigor and Pragmatism

Quantitative vs. Qualitative Outcomes: A Longitudinal Study of Risk and Ambiguity in Monetary and Medical Decision-Making