Abstract:Background: A peer-reviewed meta-analysis methods article mathematically proved that mainstream random-effects methods, "weights inversely proportional to the estimated variance," are flawed and can lead to faulty public health recommendations. Because the arguments causing this off-label (unproven) use of mainstream practices were subtle, changing these practices will require much clearer explanations that can be grasped by clinical and translational scientists. There are five assumptions underlying the mainstream's derivation of its statistical properties. This paper will demonstrate that if the first is true, it follows that the last two are false. Ratio estimation, borrowed from classical survey sampling, provides a rigorous alternative. Papers reporting results rarely fully disclose these assumptions. This is analogous to watching TV ads with the sound muted. You see high quality of life and do not hear about the complications. This article is a poster child for translational science, as it takes a theoretical discovery from the biostatistical world, translates it into language clinical scientists can understand, and thereby can change their research practice. Aim: This article is aimed at future applications of meta-analysis of complete collections of randomized clinical trials. It leaves it to past authors as to whether to reanalyze their data. No blame for past use is assessed. Methods: By treating the individual completed studies in the meta-analysis as a random sample from a conceptual universe of completed studies, we use ratio estimation to obtain estimates of relative risk (ratio of failure rates treatment: control) and mean differences, projecting our sample value to estimate the universe's value. Results: Two examples demonstrate that the mainstream methods likely adversely impacted major treatment options. A third example shows that the key mainstream presumption of independence between the study weights and study estimates cannot be supported. Conclusion: There is no rationale for ever using the mainstream for meta-analysis of randomized clinical trials. Relevance for patients: Future meta-analysis of clinical trials should never employ mainstream methods. Doing so could lead to potentially harmful public health policy recommendations. Clinical researchers need to play a primary role to assure good research practices in meta-analysis.

Problematic meta-analyses: Bayesian and frequentist perspectives on combining randomized controlled trials and non-randomized studies

Integrating randomized controlled trials and non-randomized studies of interventions to assess the effect of rare events: a Bayesian re-analysis of two meta-analyses

A Bayesian Bias-Adjusted Random-Effects Model for Synthesizing Evidence from Randomized Controlled Trials and Nonrandomized Studies of Interventions.

BAYESIAN HIERARCHICAL RANDOM-EFFECTS META-ANALYSIS AND DESIGN OF PHASE I CLINICAL TRIALS

Bayesian meta‐analysis for evaluating treatment effectiveness in biomarker subgroups using trials of mixed patient populations

Meta‐Analysis of Studies with Missing Data

Bayesian meta-analysis for evaluating treatment effectiveness in biomarker subgroups using trials of mixed patient populations

Conflating marginal and conditional treatment effects: Comments on 'Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study'

A Primer on Bayesian Model-Averaged Meta-Analysis

Comparing various Bayesian random‐effects models for pooling randomized controlled trials with rare events

Nonparametric Bayesian Approach to Treatment Ranking in Network Meta-Analysis with Application to Comparisons of Antidepressants

Comparison of statistical methods for integrating real‐world evidence in a rare events meta‐analysis of randomized controlled trials

Causal inference methods for combining randomized trials and observational studies: a review

A review and comparison of arm-based versus contrast-based network meta-analysis for binary outcomes-Understanding their differences and limitations

Random-Effect Meta-Analysis with Robust Between-Study Variance

Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis

A robust Bayesian bias-adjusted random effects model for consideration of uncertainty about bias terms in evidence synthesis

A Bayesian nonparametric approach for evaluating the causal effect of treatment in randomized trials with semi-competing risks

Meta-analysis of clinical trials in the 2020s and beyond: a paradigm shift needed

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials

Random-effects Meta-Analysis of Combined Outcomes Based on Reconstructions of Individual Patient Data