Should meta-analyses trump observational studies?
Stephanie M. Chang
DOI: https://doi.org/10.3945/ajcn.112.055111
IF: 8.472
2013-02-01
American Journal of Clinical Nutrition
Abstract:Macpherson et al (1) conducted a meta-analysis of multivitaminmultimineral (MVMM) supplementation trials to determine the effect on mortality when used for primary or secondary prevention. The underlying context was concern over whether multivitamin use is safe for those taking multivitamins for health maintenance (particularly older individuals). Further questions have been raised by some observational studies that found certain population subgroups (eg, elderly women) have an increased mortality with MVMM supplementation (2, 3). To address the safety of multivitamins for generally healthy persons who take multivitamins for health care maintenance (and not for disease treatment), Macpherson et al excluded studies in institutionalized individuals or those with a terminal illness, limiting the meta-analysis to studies of primary or secondary prevention. The authors defined MVMM as a supplement containing 3 micronutrients (except when supplements contained 3 B vitamins only), which limited studies to those using formulations most similar to commonly used commercially available products. To ensure that differences were due to the effect of MVMMs, the authors included only randomized controlled trials (RCTs) and stipulated that the oral MVMM supplements must be provided as monotherapy in one arm of the study. Because they were studying mortality as an outcome, they required that individuals be supplemented daily for at least 1 y. They also excluded studies without any deaths from the analysis. The authors did not find support for increased safety risks when independently living adults use MVMM supplements. They found, in fact, an RR of 1.00 (95% CI: 0.95, 1.06) from 20 RCTs, indicating that MVMM supplementation had no effect on mortality in either direction. How might we explain this seeming contradiction with the reports from observational studies? One explanation, made by the authors, is to point to inherent biases associated with observational studies and the added risks of multiplicity (ie, ‘‘data dredging’’). It is certainly true that design limitations from observational studies can result in spurious findings, especially given the risks of confounding and multiplicity. However, before completely discounting the findings from the observational studies in favor of meta-analyses from RCTs, we must also consider the limitations of RCTs in nutrition research. Unlike drug interventions, nutrition studies can never completely ‘‘control,’’ or even ascertain, the amount of the nutrient under investigation. This inability to ‘‘control’’ the total amount of the nutrient introduces challenges in the conduct and interpretation of RCTs in nutritional research. One common concern with meta-analyses is that they report average estimates of effect that may or may not apply to all individuals or situations. In this case, Macpherson et al did not look solely at studies with the greatest applicability to the subgroups of noted concern but targeted the generally healthy adult population who may take MVMMs to improve health and prevent disease. One might argue that some of the subpopulations identified in the observational studies (eg, elderly women or male smokers) might have differing nutrient needs or baseline nutrition status, which might alter the effectiveness or risks of MVMM supplementation. On the other hand, one might also argue that these populations are at higher risk of death overall and thus the observational studies may be confounded. In this review, the trials included individuals aged up to 86 y (average age: 61 y) with a balanced distribution of men and women. Macpherson et al found no evidence of heterogeneity between studies, although their Table 1 did not identify any studies limited to populations described as of particular concern (eg, elderly women). Unfortunately, this means that short of an individual patient-level-data meta-analysis, it is impossible to assess whether the effect estimate holds across all subpopulations included within studies. Another concern with RCTs (and syntheses of RCTs) is that they may lack the power to identify differences over long time periods. Because of resource constraints, RCTs are generally limited to shorter durations. The studies in this review averaged 43 mo (range: 1–7.5 y). The authors did not find any difference with meta-regression on the duration of study. However, even assuming adequate power to detect any differences, the longest trial was well short of the 19 y projected by the Iowa Women’s Health Study. It is possible that any small differences over a relatively short duration of time (ie, 1–7.5 y) may be magnified when extended over longer periods of time (ie, 19 y) This methodologically sound review and meta-analysis found that MVMM supplementation, when given daily for an average of