Selling Real-World Health Care Research to Reluctant Buyers-Evidence-Based Education or Marketing a Defective Product?
K. Fairman,F. Curtiss
DOI: https://doi.org/10.18553/jmcp.2009.15.3.294
2009-04-01
Journal of Managed Care Pharmacy
Abstract:In December 2004, the editors of BMJ announced a new policy requiring submission of a priori research protocols with manuscripts reporting the results of randomized controlled trials (RCTs). Under the new policy, RCTs would have to be registered at their outset with “a suitable trial registry,” and manuscripts lacking a registered protocol would no longer be sent to peer review. The editors explained that this decision had been made because their “experience of chasing authors for trial protocols, when we have suspected deviation in the protocol or found it hard to fathom what the authors set out to do,” had been “miserable.”1 As occasionally miserable editors, we too are familiar with the experience of seeking information from an author who is unable or unwilling to provide it; and evidence is mounting that we are not alone. As we and others have observed previously, the practices of selective reporting of study findings, publication planning, and other forms of misconduct are, sadly, reportedly endemic in health care research.2,3 Studies that use observational or “real-world” data, particularly pharmacoeconomic modeling and retrospective analyses of administrative databases, are particularly vulnerable to manipulation; it is especially easy to make post hoc changes to a planned protocol behind closed doors when only claims data and hypothetical patient populations, not prospectively studied human subjects, are involved.4 Thus, to the extant problem presented by Brixner et al. in their commentary on use of real-world data in this issue of JMCP5— that decision makers are sometimes reluctant to rely on analyses of real-world data—one reasonable response is that the most reluctant “buyers” of research may well be the best informed. After all, public denunciations of the “scandal of poor epidemiological research” by von Elm and Egger in 2004,6 and the “scandal of poor medical research” by Altman in 1994 and again in 20027,8 are well-known to anyone who has been following health care research even peripherally, and have spawned dozens of publications on how to improve a demonstrably inadequate pool of knowledge about the economic and clinical outcomes of health care interventions.2 Yet, we also know that many—perhaps even most—researchers are “playing by the rules,” endeavoring to provide accurate information, and producing high-quality work. So for a typical decision maker, the question becomes how to distinguish between accurate and inaccurate information. This is essentially the question raised by Brixner et al., and it is an important one. The proposals offered by the March 2008 participants in a roundtable discussion of real-world data, whose views are reported in the Brixner et al. commentary, merit consideration. Nonetheless, currently these ideas appear to generate more new questions than specific guidance. More troubling is the possibility that the effort to promote use of real-world data by decision makers may detract from ongoing efforts to improve the quality of information provided to them.