Beyond the two‐trials rule

Leonhard Held
DOI: https://doi.org/10.1002/sim.10055
2024-04-06
Statistics in Medicine
Abstract:The two‐trials rule for drug approval requires "at least two adequate and well‐controlled studies, each convincing on its own, to establish effectiveness." This is usually implemented by requiring two significant pivotal trials and is the standard regulatory requirement to provide evidence for a new drug's efficacy. However, there is need to develop suitable alternatives to this rule for a number of reasons, including the possible availability of data from more than two trials. I consider the case of up to three studies and stress the importance to control the partial Type‐I error rate, where only some studies have a true null effect, while maintaining the overall Type‐I error rate of the two‐trials rule, where all studies have a null effect. Some less‐known P ‐value combination methods are useful to achieve this: Pearson's method, Edgington's method and the recently proposed harmonic mean χ2 ‐test. I study their properties and discuss how they can be extended to a sequential assessment of success while still ensuring overall Type‐I error control. I compare the different methods in terms of partial Type‐I error rate, project power and the expected number of studies required. Edgington's method is eventually recommended as it is easy to implement and communicate, has only moderate partial Type‐I error rate inflation but substantially increased project power.
public, environmental & occupational health,medicine, research & experimental,medical informatics,mathematical & computational biology,statistics & probability
What problem does this paper attempt to address?