Abstract:A striking contrast runs through the last 60 years of biopharmaceutical discovery, research, and development. Huge scientific and technological gains should have increased the quality of academic science and raised industrial R&D efficiency. However, academia faces a "reproducibility crisis"; inflation-adjusted industrial R&D costs per novel drug increased nearly 100 fold between 1950 and 2010; and drugs are more likely to fail in clinical development today than in the 1970s. The contrast is explicable only if powerful headwinds reversed the gains and/or if many "gains" have proved illusory. However, discussions of reproducibility and R&D productivity rarely address this point explicitly. The main objectives of the primary research in this paper are: (a) to provide quantitatively and historically plausible explanations of the contrast; and (b) identify factors to which R&D efficiency is sensitive. We present a quantitative decision-theoretic model of the R&D process. The model represents therapeutic candidates (e.g., putative drug targets, molecules in a screening library, etc.) within a "measurement space", with candidates' positions determined by their performance on a variety of assays (e.g., binding affinity, toxicity, in vivo efficacy, etc.) whose results correlate to a greater or lesser degree. We apply decision rules to segment the space, and assess the probability of correct R&D decisions. We find that when searching for rare positives (e.g., candidates that will successfully complete clinical development), changes in the predictive validity of screening and disease models that many people working in drug discovery would regard as small and/or unknowable (i.e., an 0.1 absolute change in correlation coefficient between model output and clinical outcomes in man) can offset large (e.g., 10 fold, even 100 fold) changes in models' brute-force efficiency. We also show how validity and reproducibility correlate across a population of simulated screening and disease models. We hypothesize that screening and disease models with high predictive validity are more likely to yield good answers and good treatments, so tend to render themselves and their diseases academically and commercially redundant. Perhaps there has also been too much enthusiasm for reductionist molecular models which have insufficient predictive validity. Thus we hypothesize that the average predictive validity of the stock of academically and industrially "interesting" screening and disease models has declined over time, with even small falls able to offset large gains in scientific knowledge and brute-force efficiency. The rate of creation of valid screening and disease models may be the major constraint on R&D productivity.

Poor quality animal studies cause clinical trials to follow false leads.

Why There Are So Many Contradicted or Exaggerated Findings in Highly-Cited Clinical Research?

Common pitfalls in preclinical cancer target validation

Small Studies May Overestimate the Effect Sizes in Critical Care Meta-Analyses: A Meta-Epidemiological Study

Preclinical pilot studies: Five common pitfalls and how to avoid them

Can testing clinical significance reduce false positive rates in randomized controlled trials? A snap review

Abstracts Often Do Not Accurately Reflect Trial Results, Study Shows

Poor Translatability of Biomedical Research Using Animals — A Narrative Review

The methodological quality of animal research in critical care: the public face of science

Cancer: Fast tracked drugs often do not improve clinical outcomes, say researchers

Transparency Rules Lead to Large Fall in Positive Trial Results, Analysis Finds.

Lost in Translation: Failure of Preclinical Studies to Accurately Predict the Effect of Regional Analgesia on Cancer Recurrence

Dogs and monkeys in preclinical drug development: the challenge of reducing and replacing

When Quality Beats Quantity: Decision Theory, Drug Discovery, and the Reproducibility Crisis

New Tool Flags Randomised Controlled Trials That May Contain False Data

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Fool’s gold? Why blinded trials are not always best

Pitfalls of Animal Study: Avicenna’s View

Analysis of animal-to-human translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications

Decatastrophizing research irreproducibility

Productivity shortfalls in drug discovery: contributions from the preclinical sciences?