Production of murine interleukin-4 and interleukin-5 by recombinant baculovirus.

H. Rodewald,J. Langhorne,K. Eichmann,J. Kupsch

DOI: https://doi.org/10.1016/0022-1759(90)90033-R

1990-09-14

Abstract:

What problem does this paper attempt to address?

From Pre-test and Post-test Probabilities to Medical Decision Making

Michelle Pistner Nixon,Farhani Momotaz,Claire Smith,Jeffrey S Smith,Mark Sendak,Christopher Polage,Justin D Silverman

DOI: https://doi.org/10.1101/2024.02.14.24302820

2024-02-17

MedRxiv

Abstract:Background: A central goal of modern evidence-based medicine is the development of simple and easy to use tools that help clinicians integrate quantitative information into medical decision-making. The Bayesian Pre-test/Post-test Probability (BPP) framework is arguably the most well known of such tools and provides a formal approach to quantify diagnostic uncertainty given the result of a medical test or the presence of a clinical sign. Yet, clinical decision-making goes beyond quantifying diagnostic uncertainty and requires that that uncertainty be balanced against the various costs and benefits associated with each possible decision. Despite increasing attention in recent years, simple and flexible approaches to quantitative clinical decision-making have remained elusive. Methods: We extend the BPP framework using concepts of Bayesian Decision Theory. By integrating cost, we can expand the BPP framework to allow for clinical decision-making. Results: We develop a simple quantitative framework for binary clinical decisions (e.g., action/inaction, treat/no-treat, test/no-test). Let p be the pre-test or post-test probability that a patient has disease. We show that r*=(1-p)/p represents a critical value called a decision boundary. In terms of the relative cost of under- to over-acting, r* represents the critical value at which action and inaction are equally optimal. We demonstrate how this decision boundary can be used at the bedside through case studies and as a research tool through a reanalysis of a recent study which found widespread misestimation of pre-test and post-test probabilities among clinicians. Conclusions: Our approach is so simple that it should be thought of as a core, yet previously overlooked, part of the BPP framework. Unlike prior approaches to quantitative clinical decision-making, our approach requires little more than a hand-held calculator, is applicable in almost any setting where the BPP framework can be used, and excels in situations where the costs and benefits associated with a particular decision are patient-specific and difficult to quantify.
Bayesian Updating and Sequential Testing: Overcoming Inferential Limitations of Screening Tests

Jacques Balayla

DOI: https://doi.org/10.48550/arXiv.2006.11641

2020-08-30

Abstract:Bayes' Theorem confers inherent limitations on the accuracy of screening tests as a function of disease prevalence. We have shown in previous work that a testing system can tolerate significant drops in prevalence, up until a certain well-defined point known as the $prevalence$ $threshold$, below which the reliability of a positive screening test drops precipitously. Herein, we establish a mathematical model to determine whether sequential testing overcomes the aforementioned Bayesian limitations and thus improves the reliability of screening tests. We show that for a desired positive predictive value of $\rho$ that approaches $k$, the number of positive test iterations $n_i$ needed is: $ n_i =\lim_{\rho \to k}\left\lceil\frac{ln\left[\frac{\rho(\phi-1)}{\phi(\rho-1)}\right]}{ln\left[\frac{a}{1-b}\right]}\right\rceil$ where $n_i$ = number of testing iterations necessary to achieve $\rho$, the desired positive predictive value, a = sensitivity, b = specificity, $\phi$ = disease prevalence and $k$ = constant. Based on the aforementioned derivation, we provide reference tables for the number of test iterations needed to obtain a $\rho(\phi)$ of 50, 75, 95 and 99$\%$ as a function of various levels of sensitivity, specificity and disease prevalence.

Methodology
Probability bound analysis: A novel approach for quantifying parameter uncertainty in decision-analytic modeling and cost-effectiveness analysis

Rowan Iskandar

DOI: https://doi.org/10.1002/sim.9195

2021-08-12

Abstract:Decisions about health interventions are often made using limited evidence. Mathematical models used to inform such decisions often include uncertainty analysis to account for the effect of uncertainty in the current evidence base on decision-relevant quantities. However, current uncertainty quantification methodologies, including probabilistic sensitivity analysis (PSA), require modelers to specify a precise probability distribution to represent the uncertainty of a model parameter. This study introduces a novel approach for propagating parameter uncertainty, probability bounds analysis (PBA), where the uncertainty about the unknown probability distribution of a model parameter is expressed in terms of an interval bounded by lower and upper bounds on the unknown cumulative distribution function (p-box) and without assuming a particular form of the distribution function. We give the formulas of the p-boxes for common situations (given combinations of data on minimum, maximum, median, mean, or standard deviation), describe an approach to propagate p-boxes into a black-box mathematical model, and introduce an approach for decision-making based on the results of PBA. We demonstrate the characteristics and utility of PBA versus PSA using two case studies. In sum, this study provides modelers with practical tools to conduct parameter uncertainty quantification given the constraints of available data and with the fewest assumptions.

Applications,Methodology
Predictive Probabilities Made Simple: A Fast and Accurate Method for Clinical Trial Decision Making

Joe Marion,Liz Lorenzi,Cora Allen-Savietta,Scott Berry,Kert Viele

DOI: https://doi.org/10.48550/arxiv.2406.11406

2024-06-17

Applications

Abstract:Bayesian predictive probabilities are commonly used for interim monitoring of clinical trials through efficacy and futility stopping rules. Despite their usefulness, calculation of predictive probabilities, particularly in pre-experiment trial simulation, can be a significant challenge. We introduce an approximation for computing predictive probabilities using either a p-value or a posterior probability that significantly reduces this burden. We show the approximation has a high degree of concordance with standard Monte Carlo imputation methods for computing predictive probabilities, and present five simulation studies comparing the approximation to the full predictive probability for a range of primary analysis strategies: dichotomous, time-to-event, and ordinal endpoints, as well as historical borrowing and longitudinal modeling. We find that this faster method of predictive probability approximation works well in all five applications, thus significantly reducing the computational burden of trial simulation, allowing more virtual trials to be simulated to achieve greater precision in estimating trial operating characteristics.
Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers

Ahtisham Younas

DOI: https://doi.org/10.1111/jep.14032

2024-06-03

Journal of Evaluation in Clinical Practice

Abstract:Rationale Hypothesis testing is integral to health research and is commonly completed through frequentist statistics focused on computing p values. p Values have been long criticized for offering limited information about the relationship of variables and strength of evidence concerning the plausibility, presence and certainty of associations among variables. Bayesian statistics is a potential alternative for inference‐making. Despite emerging discussion on Bayesian statistics across various disciplines, the uptake of Bayesian statistics in health research is still limited. Aim To offer a primer on Bayesian statistics and Bayes factors for health researchers to gain preliminary knowledge of its use, application and interpretation in health research. Methods Theoretical and empirical literature on Bayesian statistics and methods were used to develop this methodological primer. Conclusions Using Bayesian statistics in health research without a careful and complete understanding of its underlying philosophy and differences from frequentist testing, estimation and interpretation methods can result in similar ritualistic use as done for p values. Implications Health researchers should supplement frequentists statistics with Bayesian statistics when analysing research data. The overreliance on p values for clinical decisions making should be avoided. Bayes factors offer a more intuitive measure of assessing the strength of evidence for null and alternative hypothesis.

medicine, general & internal,health care sciences & services,medical informatics
Decision Curve Analysis: a Technical Note

Zhongheng Zhang,Valentin Rousson,Wen-Chung Lee,Cyril Ferdynus,Mingyu Chen,Xin Qian,Yizhan Guo

DOI: https://doi.org/10.21037/atm.2018.07.02

IF: 3.616

2018-01-01

Annals of Translational Medicine

Abstract:Multivariable regression models are widely used in medical literature for the purpose of diagnosis or prediction. Conventionally, the adequacy of these models is assessed using metrics of diagnostic performances such as sensitivity and specificity, which fail to account for clinical utility of a specific model. Decision curve analysis (DCA) is a widely used method to measure this utility. In this framework, a clinical judgment of the relative value of benefits (treating a true positive case) and harms (treating a false positive case) associated with prediction models is made. As such, the preferences of patients or policy-makers are accounted for by using a metric called threshold probability. A decision analytic measure called net benefit is then calculated for each possible threshold probability, which puts benefits and harms on the same scale. The article is a technical note on how to perform DCA in R environment. The decision curve is depicted with the ggplot2 system. Correction for overfitting is done via either bootstrap or cross-validation. Confidence interval and P values for the comparison of two models are calculated using bootstrap method. Furthermore, we describe a method for computing area under net benefit for the comparison of two models. The average deviation about the probability threshold (ADAPT), which is a more recently developed index to measure the utility of a prediction model, is also introduced in this article.
A Bayesian Framework for Patient-Level Partitioned Survival Cost-Utility Analysis

Andrea Gabrio

DOI: https://doi.org/10.1177/0272989X211012348

2021-05-21

Medical Decision Making

Abstract:Patient-level health economic data collected alongside clinical trials are an important component of the process of technology appraisal. For end-of-life treatments, the modeling of cost-effectiveness data may involve some form of partitioned survival analysis, in which measures of quality of life and survival for pre- and postprogression periods are combined to generate aggregate measures of clinical benefits (e.g., quality-adjusted survival). In addition, resource use data are often collected and costs are calculated for each type of health service (e.g., treatment, hospital, or adverse events costs). A critical problem in these analyses is that effectiveness and cost data present some complexities, such as nonnormality, spikes, and missingness, which should be addressed using appropriate methods to avoid biased results. This article proposes a general Bayesian framework that takes into account the complexities of trial-based partitioned survival cost-utility data to provide more adequate evidence for policy makers. Our approach is motivated by, and applied to, a working example based on data from a trial assessing the cost-effectiveness of a new treatment for patients with advanced non–small-cell lung cancer. Highlights • This is the first article proposing a Bayesian modeling framework for patient-level partitioned survival cost-utility analysis that can jointly take into account the typical complexities of the data, such as correlation, skewness, and spikes at zeros. • The framework is defined using a modular structure that enables a flexible model specification in terms of a sequence of conditional parametric distributions that can be chosen based on the specific characteristics of each modeled variable (e.g., type of effectiveness or cost component). • The proposed methods overcome the limitations of standard approaches that ignore at least some of the complexities of the data and, although easier to implement and well established among practitioners, may mislead cost-effectiveness decisions of policy makers. Social media content • Draft Tweet: The article proposes a new modeling Bayesian framework for trial-based partitioned survival cost-utility analysis that can flexibly take into account the impact of different sources of uncertainty on cost-effectiveness conclusions • Twitter's account name: Andrea Gabrio (@AndreaGabrio) • Preferred tags: #CostEffectiveness; #BayesianStatistics, #EconomicEvaluation

health care sciences & services,medical informatics,health policy & services
IAPN: a simple framework for evaluating whether a population-based risk stratification tool will be cost-effective before implementation

Steven Wyatt,Mohammed A. Mohammed,Peter Spilsbury

DOI: https://doi.org/10.1186/s12962-024-00594-5

2024-12-06

Cost Effectiveness and Resource Allocation

Abstract:Risk prediction tools are widely used in healthcare to identify individuals at high risk of adverse events who may benefit from proactive interventions. Traditionally, these tools are evaluated primarily on statistical performance measures—such as sensitivity, specificity, discrimination, and positive predictive value (PPV)—with minimal attention given to their cost-effectiveness. As a result, while many published tools report high performance statistics, evidence is limited on their real-world efficacy and potential for cost savings. To address this gap, we propose a straightforward framework for evaluating risk prediction tools during the design phase, which incorporates both PPV and intervention effectiveness, measured by the number needed to treat (NNT). This framework shows that to be cost-effective, the per-unit cost of an intervention (I) must be less than the average cost of the adverse event (A) multiplied by the PPV-to-NNT ratio: I < A*PPV/NNT. This criterion enables decision-makers to assess the economic value of a risk prediction tool before implementation.

health policy & services
Bayesian Decision Curve Analysis With Bayesdca

Giuliano Netto Flores Cruz,Keegan Korthauer

DOI: https://doi.org/10.1002/sim.10277

2024-12-03

Statistics in Medicine

Abstract:Clinical decisions are often guided by clinical prediction models or diagnostic tests. Decision curve analysis (DCA) combines classical assessment of predictive performance with the consequences of using these strategies for clinical decision‐making. In DCA, the best decision strategy is the one that maximizes the net benefit: the net number of true positives (or negatives) provided by a given strategy. Here, we employ Bayesian approaches to DCA, addressing four fundamental concerns when evaluating clinical decision strategies: (i) which strategies are clinically useful, (ii) what is the best available decision strategy, (iii) which of two competing strategies is better, and (iv) what is the expected net benefit loss associated with the current level of uncertainty. While often consistent with frequentist point estimates, fully Bayesian DCA allows for an intuitive probabilistic interpretation framework and the incorporation of prior evidence. We evaluate the methods using simulation and provide a comprehensive case study. Software implementation is available in the bayesDCA R package. Ultimately, the Bayesian DCA workflow may help clinicians and health policymakers adopt better‐informed decisions.

public, environmental & occupational health,medicine, research & experimental,medical informatics,mathematical & computational biology,statistics & probability
Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research

Riko Kelter

DOI: https://doi.org/10.1186/s12874-020-00968-2

2020-04-22

BMC Medical Research Methodology

Abstract:Abstract Background The replication crisis hit the medical sciences about a decade ago, but today still most of the flaws inherent in null hypothesis significance testing (NHST) have not been solved. While the drawbacks of p -values have been detailed in endless venues, for clinical research, only a few attractive alternatives have been proposed to replace p -values and NHST. Bayesian methods are one of them, and they are gaining increasing attention in medical research, as some of their advantages include the description of model parameters in terms of probability, as well as the incorporation of prior information in contrast to the frequentist framework. While Bayesian methods are not the only remedy to the situation, there is an increasing agreement that they are an essential way to avoid common misconceptions and false interpretation of study results. The requirements necessary for applying Bayesian statistics have transitioned from detailed programming knowledge into simple point-and-click programs like JASP. Still, the multitude of Bayesian significance and effect measures which contrast the gold standard of significance in medical research, the p -value, causes a lack of agreement on which measure to report. Methods Therefore, in this paper, we conduct an extensive simulation study to compare common Bayesian significance and effect measures which can be obtained from a posterior distribution. In it, we analyse the behaviour of these measures for one of the most important statistical procedures in medical research and in particular clinical trials, the two-sample Student’s (and Welch’s) t-test. Results The results show that some measures cannot state evidence for both the null and the alternative. While the different indices behave similarly regarding increasing sample size and noise, the prior modelling influences the obtained results and extreme priors allow for cherry-picking similar to p-hacking in the frequentist paradigm. The indices behave quite differently regarding their ability to control the type I error rates and regarding their ability to detect an existing effect. Conclusion Based on the results, two of the commonly used indices can be recommended for more widespread use in clinical and biomedical research, as they improve the type I error control compared to the classic two-sample t-test and enjoy multiple other desirable properties.

health care sciences & services
Influence of the Way Results Are Presented on Research Interpretation and Medical Decision Making: the PRIMER Collaboration Randomized Studies

Thomas Philip Shakespeare,Val Gebski,Johann Tang,Keith Lim,Jiade Jay Lu,Xiaojian Zhang,Guoliang Jiang

DOI: https://doi.org/10.1177/0272989x07309640

2007-01-01

Medical Decision Making

Abstract:Background. The manner of presentation of research results may affect how clinicians interpret research and make clinical decisions. The authors evaluate whether the use of confidence levels improve research interpretation and decision making compared with P values and 95% confidence intervals. Methods. The 2 Presentation and Interpretation of Medical Research (PRIMER) studies were 3-arm randomized trials. PRIMER 1 presented results of 5 fictitious scenarios with P values (P), P plus 95% confidence intervals (P + CI), or P, CI, and confidence levels (P + CI + CL); PRIMER 2 compared P + CI + CL, P + CI, and P + CL. Clinicians were asked to identify the correct interpretation of scenarios in terms of statistical and clinical significance and then indicate the intended decision making in terms of treatment recommendation. Results. Seventy-five and 246 clinicians participated in PRIMER 1 and PRIMER 2, respectively. In PRIMER 1, P+CI+CL was superior to P + CI and P (P < 0.05); the latter 2 arms did not differ significantly. Decision making was not significantly different between arms. In PRIMER 2, P+CI+CL resulted in better interpretation than P + CI (P = 0.03), with no difference between P + CI and P + CL. In combined analysis, the odds of correct interpretation were higher for P+CI+CL than P+CI (odds ratio = 1.73, P=0.005, 95% CI= 1.19--2.52). Decision making was better for P + CI+ CL (P = 0.03). On multivariate analysis, the P + CI+ CL arm and clinicians with statistics training, not in private practice, or participating in PRIMER 1 had better interpretation. The P + CI+ CL arm was the only factor improving decision making. Conclusions. Presenting research with a combination of P values, confidence intervals, and confidence levels leads to better interpretation and decision making by clinicians.
Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP

Riko Kelter

DOI: https://doi.org/10.1186/s12874-020-00980-6

2020-06-05

BMC Medical Research Methodology

Abstract:Abstract Background Although null hypothesis significance testing (NHST) is the agreed gold standard in medical decision making and the most widespread inferential framework used in medical research, it has several drawbacks. Bayesian methods can complement or even replace frequentist NHST, but these methods have been underutilised mainly due to a lack of easy-to-use software. JASP is an open-source software for common operating systems, which has recently been developed to make Bayesian inference more accessible to researchers, including the most common tests, an intuitive graphical user interface and publication-ready output plots. This article provides a non-technical introduction to Bayesian hypothesis testing in JASP by comparing traditional tests and statistical methods with their Bayesian counterparts. Results The comparison shows the strengths and limitations of JASP for frequentist NHST and Bayesian inference. Specifically, Bayesian hypothesis testing via Bayes factors can complement and even replace NHST in most situations in JASP. While p -values can only reject the null hypothesis, the Bayes factor can state evidence for both the null and the alternative hypothesis, making confirmation of hypotheses possible. Also, effect sizes can be precisely estimated in the Bayesian paradigm via JASP. Conclusions Bayesian inference has not been widely used by now due to the dearth of accessible software. Medical decision making can be complemented by Bayesian hypothesis testing in JASP, providing richer information than single p -values and thus strengthening the credibility of an analysis. Through an easy point-and-click interface researchers used to other graphical statistical packages like SPSS can seemlessly transition to JASP and benefit from the listed advantages with only few limitations.

health care sciences & services
A Software Tool for Estimating Uncertainty of Bayesian Posterior Probability for Disease

Theodora Chatzimichail,Aristides T. Hatjimihail

DOI: https://doi.org/10.3390/diagnostics14040402

IF: 3.6

2024-02-13

Diagnostics

Abstract:The role of medical diagnosis is essential in patient care and healthcare. Established diagnostic practices typically rely on predetermined clinical criteria and numerical thresholds. In contrast, Bayesian inference provides an advanced framework that supports diagnosis via in-depth probabilistic analysis. This study's aim is to introduce a software tool dedicated to the quantification of uncertainty in Bayesian diagnosis, a field that has seen minimal exploration to date. The presented tool, a freely available specialized software program, utilizes uncertainty propagation techniques to estimate the sampling, measurement, and combined uncertainty of the posterior probability for disease. It features two primary modules and fifteen submodules, all designed to facilitate the estimation and graphical representation of the standard uncertainty of the posterior probability estimates for diseased and non-diseased population samples, incorporating parameters such as the mean and standard deviation of the test measurand, the size of the samples, and the standard measurement uncertainty inherent in screening and diagnostic tests. Our study showcases the practical application of the program by examining the fasting plasma glucose data sourced from the National Health and Nutrition Examination Survey. Parametric distribution models are explored to assess the uncertainty of Bayesian posterior probability for diabetes mellitus, using the oral glucose tolerance test as the reference diagnostic method.

medicine, general & internal
A unified Bayesian framework for interval hypothesis testing in clinical trials

Abhisek Chakraborty,Megan H. Murray,Ilya Lipkovich,Yu Du

2024-02-22

Abstract:The American Statistical Association (ASA) statement on statistical significance and P-values \cite{wasserstein2016asa} cautioned statisticians against making scientific decisions solely on the basis of traditional P-values. The statement delineated key issues with P-values, including a lack of transparency, an inability to quantify evidence in support of the null hypothesis, and an inability to measure the size of an effect or the importance of a result. In this article, we demonstrate that the interval null hypothesis framework (instead of the point null hypothesis framework), when used in tandem with Bayes factor-based tests, is instrumental in circumnavigating the key issues of P-values. Further, we note that specifying prior densities for Bayes factors is challenging and has been a reason for criticism of Bayesian hypothesis testing in existing literature. We address this by adapting Bayes factors directly based on common test statistics. We demonstrate, through numerical experiments and real data examples, that the proposed Bayesian interval hypothesis testing procedures can be calibrated to ensure frequentist error control while retaining their inherent interpretability. Finally, we illustrate the improved flexibility and applicability of the proposed methods by providing coherent frameworks for competitive landscape analysis and end-to-end Bayesian hypothesis tests in the context of reporting clinical trial outcomes.

Machine Learning,Methodology
Weighted Brier Score -- an Overall Summary Measure for Risk Prediction Models with Clinical Utility Consideration

Kehao Zhu,Yingye Zheng,Kwun Chuen Gary Chan

DOI: https://doi.org/10.48550/arxiv.2408.01626

2024-08-03

Methodology

Abstract:As advancements in novel biomarker-based algorithms and models accelerate disease risk prediction and stratification in medicine, it is crucial to evaluate these models within the context of their intended clinical application. Prediction models output the absolute risk of disease; subsequently, patient counseling and shared decision-making are based on the estimated individual risk and cost-benefit assessment. The overall impact of the application is often referred to as clinical utility, which received significant attention in terms of model assessment lately. The classic Brier score is a popular measure of prediction accuracy; however, it is insufficient for effectively assessing clinical utility. To address this limitation, we propose a class of weighted Brier scores that aligns with the decision-theoretic framework of clinical utility. Additionally, we decompose the weighted Brier score into discrimination and calibration components, examining how weighting influences the overall score and its individual components. Through this decomposition, we link the weighted Brier score to the $H$ measure, which has been proposed as a coherent alternative to the area under the receiver operating characteristic curve. This theoretical link to the $H$ measure further supports our weighting method and underscores the essential elements of discrimination and calibration in risk prediction evaluation. The practical use of the weighted Brier score as an overall summary is demonstrated using data from the Prostate Cancer Active Surveillance Study (PASS).
Novel decision-theoretic and risk-stratification metrics of predictive performance: Application to deciding who should undergo genetic testing

Hormuzd A. Katki

DOI: https://doi.org/10.48550/arXiv.1711.05686

2017-11-16

Abstract:Currently, women are referred for BRCA1/2 mutation-testing only if their family-history of breast/ovarian cancer implies that their risk of carrying a mutation exceeds 10\%. However, as mutation-testing costs fall, prominent voices have called for testing all women, which would strain clinical resources by testing millions of women, almost all of whom will test negative. To better evaluate risk-thresholds for BRCA1/2 testing, we introduce two broadly applicable, linked metrics: Mean Risk Stratification (MRS) and a decision-theoretic metric, Net Benefit of Information (NBI). MRS and NBI provide a range of risk thresholds at which a marker/model is "optimally informative", in the sense of maximizing both MRS and NBI. NBI is a function of only MRS and the risk-threshold for action, connecting decision-theory to risk-stratification and providing a decision-theoretic rationale for MRS. AUC and Youden's index reflect on both the fraction of maximum MRS, and of maximum NBI, attained by the marker/model, providing AUC and Youden's index with long-sought decision-theoretic and risk-stratification rationale. To evaluate risk-thresholds for BRCA1/2 testing, we propose an eclectic approach considering AUC, Net Benefit, and MRS/NBI. MRS/NBI interpret AUC in the context of mutation-prevalence and provide a range of risk thresholds for which the risk model is optimally informative.

Methodology,Applications
A Utilitarian Perspective on Risk Quantification for Clinical Significance in Binary Outcomes

Junhui Park

DOI: https://doi.org/10.1177/00469580241248134

2024-01-01

Abstract:Null hypothesis significance testing (NHST) in medical research is increasingly being supplemented by estimation statistics, focusing on effect sizes (ESs) and confidence intervals (CIs). This study evaluates the expression of ESs and CIs for binary outcomes. A utilitarian framework is proposed, emphasizing the number of beneficiaries and the impact level. To evaluate clinical significance, minimal clinically important risk difference (MCIRD) is proposed based on event magnitude (EM). Within this framework, risk difference (RD) is introduced as the primary measure. To assess the performance of RD, we compared its statistical power against other measures (risk ratio, RR; odds ratio, OR; Cohen’s h) in individual study scenarios, and visual information conveyance in meta-analysis scenarios. RDs maintain statistical power in comparison to other measures in individual studies. They provide clarity on the true impact of clinical interventions without compromising statistical integrity. Meta-analytic results indicate that using RDs directly enhances transparency, uncovers heterogeneity, and addresses misaligned assumptions. This approach, by quantifying clinical effectiveness under a utilitarian perspective, facilitates the applicability of research to patient care and encourages shared decision-making. The study advocates for reporting baseline risks (BRs) with RDs and recommends a standardized presentation of these statistics. In a utilitarian perspective, adopting RD as the preferred ES can foster a transparent, patient-focused research ethos. This aids in accurately presenting the magnitude and variability of treatment effects, offering a new direction in methodology.

health care sciences & services,health policy & services
Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

Andrew J Vickers,Ben Van Calster,Ewout W Steyerberg

DOI: https://doi.org/10.1136/bmj.i6

2016-01-25

BMJ

Abstract:Many decisions in medicine involve trade-offs, such as between diagnosing patients with disease versus unnecessary additional testing for those who are healthy. Net benefit is an increasingly reported decision analytic measure that puts benefits and harms on the same scale. This is achieved by specifying an exchange rate, a clinical judgment of the relative value of benefits (such as detecting a cancer) and harms (such as unnecessary biopsy) associated with models, markers, and tests. The exchange rate can be derived by asking simple questions, such as the maximum number of patients a doctor would recommend for biopsy to find one cancer. As the answers to these sorts of questions are subjective, it is possible to plot net benefit for a range of reasonable exchange rates in a “decision curve.” For clinical prediction models, the exchange rate is related to the probability threshold to determine whether a patient is classified as being positive or negative for a disease. Net benefit is useful for determining whether basing clinical decisions on a model, marker, or test would do more good than harm. This is in contrast to traditional measures such as sensitivity, specificity, or area under the curve, which are statistical abstractions not directly informative about clinical value. Recent years have seen an increase in practical applications of net benefit analysis to research data. This is a welcome development, since decision analytic techniques are of particular value when the purpose of a model, marker, or test is to help doctors make better clinical decisions.
A Tutorial on Modern Bayesian Methods in Clinical Trials

Natalia Muehlemann,Tianjian Zhou,Rajat Mukherjee,Munshi Imran Hossain,Satrajit Roychoudhury,Estelle Russek-Cohen

DOI: https://doi.org/10.1007/s43441-023-00515-3

2023-04-21

Abstract:Clinical trials continue to be the gold standard for evaluating new medical technologies. New advancements in modern computation power have led to increasing interest in Bayesian methods. Despite the multiple benefits of Bayesian approaches, application to clinical trials has been limited. Based on insights from the survey of clinical researchers in drug development conducted by the Drug Information Association Bayesian Scientific Working Group (DIA BSWG), insufficient knowledge of Bayesian approaches was ranked as the most important perceived barrier to implementing Bayesian methods. Results of the same survey indicate that clinical researchers may find the interpretation of results from a Bayesian analysis to be more useful than conventional interpretations. In this article, we illustrate key concepts tied to Bayesian methods, starting with familiar concepts widely used in clinical practice before advancing in complexity, and use practical illustrations from clinical development.

pharmacology & pharmacy,medical informatics
A puzzle of proportions: Two popular Bayesian tests can yield dramatically different conclusions

Fabian Dablander,Karoline Huth,Quentin F. Gronau,Alexander Etz,Eric‐Jan Wagenmakers

DOI: https://doi.org/10.1002/sim.9278

2021-12-12

Statistics in Medicine

Abstract:Testing the equality of two proportions is a common procedure in science, especially in medicine and public health. In these domains, it is crucial to be able to quantify evidence for the absence of a treatment effect. Bayesian hypothesis testing by means of the Bayes factor provides one avenue to do so, requiring the specification of prior distributions for parameters. The most popular analysis approach views the comparison of proportions from a contingency table perspective, assigning prior distributions directly to the two proportions. Another, less popular approach views the problem from a logistic regression perspective, assigning prior distributions to logit-transformed parameters. Reanalyzing 39 null results from the New England Journal of Medicine with both approaches, we find that they can lead to markedly different conclusions, especially when the observed proportions are at the extremes (ie, very low or very high). We explain these stark differences and provide recommendations for researchers interested in testing the equality of two proportions and users of Bayes factors more generally. The test that assigns prior distributions to logit-transformed parameters creates prior dependence between the two proportions and yields weaker evidence when the observations are at the extremes. When comparing two proportions, we argue that this test should become the new default.

public, environmental & occupational health,medicine, research & experimental,medical informatics,mathematical & computational biology,statistics & probability

Production of murine interleukin-4 and interleukin-5 by recombinant baculovirus.

From Pre-test and Post-test Probabilities to Medical Decision Making

Bayesian Updating and Sequential Testing: Overcoming Inferential Limitations of Screening Tests

Probability bound analysis: A novel approach for quantifying parameter uncertainty in decision-analytic modeling and cost-effectiveness analysis

Predictive Probabilities Made Simple: A Fast and Accurate Method for Clinical Trial Decision Making

Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers

Decision Curve Analysis: a Technical Note

A Bayesian Framework for Patient-Level Partitioned Survival Cost-Utility Analysis

IAPN: a simple framework for evaluating whether a population-based risk stratification tool will be cost-effective before implementation

Bayesian Decision Curve Analysis With Bayesdca

Analysis of Bayesian posterior significance and effect size indices for the two-sample t-test to support reproducible medical research

Influence of the Way Results Are Presented on Research Interpretation and Medical Decision Making: the PRIMER Collaboration Randomized Studies

Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP

A Software Tool for Estimating Uncertainty of Bayesian Posterior Probability for Disease

A unified Bayesian framework for interval hypothesis testing in clinical trials

Weighted Brier Score -- an Overall Summary Measure for Risk Prediction Models with Clinical Utility Consideration

Novel decision-theoretic and risk-stratification metrics of predictive performance: Application to deciding who should undergo genetic testing

A Utilitarian Perspective on Risk Quantification for Clinical Significance in Binary Outcomes

Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

A Tutorial on Modern Bayesian Methods in Clinical Trials

A puzzle of proportions: Two popular Bayesian tests can yield dramatically different conclusions