Should studies with no events in both arms be excluded in evidence synthesis?

Chang Xu,Luis Furuya-Kanamori,Nazmul Islam,Suhail A. Doi

DOI: https://doi.org/10.1016/j.cct.2022.106962

IF: 2.261

2022-01-01

Contemporary Clinical Trials

Abstract:Objectives: In safety assessment, studies with no events are a frequent occurrence when conducting meta -analyses. The current approach in meta-analysis is to exclude double-zero studies from the synthesis. In this study, we compared the performance of excluding and including double-zero studies.Methods: A simulation with 5000 iterations was conducted based on the real-world dataset from Cochrane re-views. The true distribution of the rare events rather than normal distribution for the effects were used in the data generating mechanism to simulate aggregate meta-analysis data. We used Doi's inverse variance hetero-geneity (IVhet) model for the meta-analyses with continuity correction (of 0.5) to include double-zero studies and used the odds ratio effect size. The performance of including versus excluding double-zero studies were then compared.Results: Generally, there was much larger mean squared error when double zero studies were excluded than when double-zero studies were included. The coverage when studies were excluded rapidly deteriorates as heteroge-neity increased, while remained at or above the nominal level when double-zero studies were included. When there were very few double-zero studies, the performances was almost the same when including or excluding these studies. Subgroup analysis showed that, even for meta-analyses with unbalanced sample size across the two arms, including double-zero studies improved performance compared to when they were excluded.Conclusions: Including double-zero studies in meta-analysis improved performance substantively when compared to excluding them, especially when the proportion of double-zero studies was large. Continuity correction with use of the IVhet model is therefore a good solution to deal with double-zero studies and should be considered in future meta-analyses.