What is the Evidence Base for the Randomized Controlled Trial?
Zachary Bloomgarden,Guang Ning
DOI: https://doi.org/10.1111/j.1753-0407.2009.00029.x
IF: 4.53
2009-01-01
Journal of Diabetes
Abstract:Michael Rawlins, in his Harveian Oration on the concept of evidence in clinical decision making, discusses a number of issues relevant to the development of optimal glycemic treatment approaches for people with diabetes.1 The ability to generalize from randomized controlled trials (RCT) of glycemic treatment may be limited. Treatments may be more effective in patients younger or older, or with longer or shorter duration of disease, or with greater or lesser severity of already existing complications, or with more or fewer comorbidities or risk factors, or with different ethnicity, or with sex differences, or socioeconomic differences, or cultural differences. Longer duration, high baseline A1c, and existing cardiovascular complications appear to be particularly associated with treatment harm in the Action to Control Cardiovascular Risk in Diabetes (ACCORD)2 and Veterans Affairs Diabetes Trial (VADT)3 trials, but other patient-specific factors may be equally important. Treatments may be given at too high or too low a dose or, perhaps relevant to the question of glycemic treatment, the metric used in assessing dose may not be sufficiently flexible to accommodate all patients. We refer here to the perhaps mistaken use of A1c as the sine qua non in determining glycemia, inasmuch as there is good evidence of heterogeneity in the degree to which different individuals glycate hemoglobin, so that a ‘high glycator’ may be excessively treated at a given level of A1c, leading to predictable side-effects of excessive glucose-lowering medication.4 The various treatment regimens used to attain control of glycemia surely cannot be assumed to be identical in effect, so that the notion of ‘glycemic strategy’, in which various classes of medications can be used at the discretion of the treating physician and then compared, must be seen as one that may obscure important differences between agents. Furthermore, with long periods of treatment, the duration of therapeutic effectiveness of various agents becomes an important issue, so that recommendations for use of sulfonylureas,5 which show good short-term but poor long-term efficacy,6 may be inappropriate. In addition, the common finding that side-effects of a given agent tend to be seen early in their use but improvements in outcome emerge only after many years of treatment is likely to lead to incorrect interpretation of risk–benefit ratios of agents intended for long-term use. Other issues include factors related to patient adherence/compliance, the potential of a given agent to show adverse interactions with comedication, and the setting in which the studied treatment is ultimately to be used. Prescription and monitoring by less expert healthcare providers may markedly change the quality of care and the potential benefit of a given approach. An important corollary of these concepts is Rawlins’ assertion that we have incorrectly elevated the RCT to a position at the peak of the hierarchy of sources of clinical knowledge.1 In addition to the issue of generalizability, he points out that devising stopping rules to end a trial early, the performance of subgroup analyses, the assessment of multiple endpoints, and the related assessment of adverse outcomes all require multiple calculations of statistical significance, leading to overestimation of likelihood. An example of this is the increase in mortality leading to termination of the ACCORD trial (Fig. 1). One presumes that multiple potential adverse outcomes were examined, at multiple time points, and the trend to subsequent benefit of the primary endpoints of cardiovascular (CV) mortality, non-fatal myocardial infarction, and non-fatal stroke, particularly with evidence that the mortality benefit of the Steno 2 intervention in diabetic patients was not seen until after more than a decade of treatment with, in fact, a trend to increased mortality at 4 years (Fig. 2),7 suggests further limitation of the RCT as the sole approach to determining appropriate glycemic treatment interventions for people with diabetes. The Action to Control Cardiovascular Risk in Diabetes Study. (a) Composite of cardiovascular mortality, non-fatal myocardial infarction, and non-fatal stroke. (b) Total mortality. The trial was discontinued at a mean follow up of 3.5 years. The suggestion of benefit emerging at 5 years was seen in <10% of the original cohort and was not of statistical significance. Reprinted with permission from Action to Control Cardiovascular Risk in Diabetes Study Group.1 Steno study,7 mortality. Note that at 3.5 years (the time of termination of ACCORD for increased mortality), mortality in the intensive therapy group was more than twice that in the conventional therapy group (6.3% vs 2.5%, respectively), although over the subsequent decade the mortality in the conventional therapy groups was approximately 60% as great as that in the intensive therapy group. Reprinted with permission from Gaede et al.7 Rawlins finally discusses the resource implications of reliance on the RCT to determine approaches to medical intervention.1 The typical 6-month trial has a cost well over US$10 000 per patient. Let us consider the potential requirements for an intensive glycemic treatment trial of early diabetes, a target many have suggested as appropriate given the suggestion of harm to people with more advanced disease in ACCORD and VADT. It appears that CV event rates in people with Type 2 diabetes who are receiving appropriate lipid, blood pressure, and antiplatelet therapies are lower than previously thought, with the relatively advanced disease patients in ACCORD only having a 5-year incidence of the primary endpoint of approximately 10%.2 Asymptomatic patients with earlier diabetes appear to have CV rates less than half as great.8 A conservative estimate may then be event rates of 6–8% at 10 years in diabetic patients receiving optimal risk factor treatment. In this setting, the efficacy of glycemic treatment in reducing CV events may be no more than 10–15%. The early diabetes trial would then require a sample size of 15 000–50 000 people, assuming a statistical power of 80%. The true cost would be no less than US$10 000 per patient per year, for a total of US$1.5–5 billion, dwarfing the cost of all prior diabetes studies. Issues of generalizability, multiple subanalyses, and appropriate statistical hypothesis testing remain. What are the alternatives? Establishing large databases with high levels of excellence in ascertainment of treatments, CV and glycemic risk factors, and medical and health economic outcomes will allow prospective acquisition of data to address basic questions required in planning appropriate treatment for the coming diabetes epidemic. Does treating early diabetes matter more than treating it later? Does glycemic treatment of early diabetes actually lead to improved outcome? Is A1c a good test for mean glucose in individual patients and should A1c be used as the ‘goal’, or should the goal be given in terms of self-monitored (or continuously monitored) glucose levels? Is hypoglycemia important in Type 2 diabetes? Do drugs with a longer duration of glycemic control lead to more favorable outcomes? Should sulfonylureas continue to be widely used or should agents less likely to cause hypoglycemia be considered despite their greater cost? The potential for human suffering is great, the costs associated with diabetes are immense, and the potential for future benefit suggests this endeavor to be of enormous importance. Of course, we should not abandon the RCT. Particularly when investigating outcomes affecting a large subset (or all) of the participants, the RCT is a sturdy approach to understanding the effect of an intervention. However, we must incorporate a more flexible concept of appropriate clinical evidence in the development of treatments for hundreds of millions of people who have, and will have, diabetes.