Kirsch Et Al.'s (2008) Calculations Are Correct: Reconsidering Fountoulakis & Möller's Re-Analysis of the Kirsch Data

Tania B. Huedo‐Medina,Blair T. Johnson,Irving Kirsch
DOI: https://doi.org/10.1017/s1461145711001878
2012-01-01
Abstract:Our meta-analysis examining the efficacy of antidepressant medication (Kirsch et al. 2008), based on trials submitted to the Food and Drug Administration (FDA) for antidepressant drug approval, concluded that clinically significant improvement in depression attributable to antidepressants as opposed to placebo appeared only for samples with exceptionally severe depression. Our criterion for clinical significance was that used by the National Institute for Health and Clinical Excellence (NICE), which establishes treatment guidelines for the National Health Service in the UK. Published critiques of our statistical approach (e.g. Horder et al . 2010; Huf et al. 2011) as well as the clinical implications of our findings (e.g. Moller & Broich, 2010) were all items raised in online commentaries when our article was published and we responded to these criticisms (Huedo-Medina et al. 2008; Johnson et al. 2008). Although disagreement has generally centred on interpretations of the data, Fountoulakis & Moller (2010) re-analysed the data and claimed that they had discovered ‘important flaws in our calculations’, although they found the same overall, standardized mean difference effect size ( d ) that we had, which was substantially below the NICE criterion for clinical significance. Their methods obtained weighted mean improvement scores on the Hamilton Rating Scale for Depression (HAMD) that were different from ours. Given the attention that our original analysis has received, we believe it is important that readers know which results are accurate. We first present in greater detail the methods we had used in obtaining those aspects of the results that are in dispute, along with the justification for using those methods. Next, we examine the methods used by Fountoulakis & Moller in their attempted replication. We followed conventional meta-analytical procedures to summarize the FDA data we used. Of particular note are the weighting of change scores on the HAMD within …
What problem does this paper attempt to address?