The importance of quality of primary studies in producing unbiased systematic reviews.

Khalid Khan,S. Daya,A. Jadad
DOI: https://doi.org/10.1001/ARCHINTE.1996.00440060089011
1996-03-25
Archives of Internal Medicine
Abstract:BACKGROUND Traditional and largely qualitative reviews of evidence are now giving way to much more structured systematic overviews that use a quantitative method to calculate the overall effect of treatment. The latter approach is dependent on the quality of primary studies, which may introduce bias if they are of poor methodologic quality. OBJECTIVE To test the hypothesis that the inclusion of poor-quality trials in meta-analyses would bias the conclusions and produce incorrect estimates of treatment effect. METHODS An overview of randomized trials of antiestrogen therapy in subfertile men with oligospermia was performed to test the hypothesis. Data sources included online searching of MEDLINE and Science Citation Index databases between 1966 and 1994, scanning the bibliography of known primary studies and review articles, and contacting experts in the field. After independent, blind assessment, nine of 149 originally identified studies met the inclusion criteria and were selected. We assessed study quality independently. Outcome data from each study were pooled and statistically summarized. RESULTS There was a marginal improvement in pregnancy rate with antiestrogen treatment (odds ratio, 1.6; 95% confidence interval, 0.9 to 2.6). Sensitivity analyses on the basis of methodologic quality demonstrated that poor-quality studies produced a positive effect with treatment, whereas no benefit was observed with high-quality studies. CONCLUSION The results of a meta-analysis are influenced by the quality of the primary studies included. Methodologically, poor studies tend to exaggerate the overall estimate of treatment effect and may lead to incorrect inferences.
What problem does this paper attempt to address?