Disproportionality analyses of spontaneous reports

S. Hennessy

DOI: https://doi.org/10.1002/pds.995

2004-08-01

Pharmacoepidemiology and Drug Safety

Abstract:Case reports and case series are widely regarded as useful for raising hypotheses to be tested in subsequent controlled studies, but generally insufficient to constitute convincing evidence of a cause-effect relationship. Nevertheless, publication of case reports is common and seems to evoke little concern that it fosters unfounded beliefs about particular cause-effect relationships. This is presumably because of the widespread recognition of the limited interpretability of case reports. Those performing pharmacovigilance sometimes perform quantitative analyses of data from spontaneous reporting systems as a supplement to qualitatively examining individual case reports. For example, one can calculate, for a given drug, the relative frequency of a specific event versus all events reported for that drug, divided by the corresponding quantity for other drugs. If the frequencies are expressed as proportions, then this ratio is the proportional reporting ratio (PRR); if expressed as odds, then the ratio is the reporting odds ratio (ROR). Collectively, the PRR and ROR are known as disproportionality measures. The relative merits of the PRR versus the ROR are discussed elsewhere. Disproportionality measures were developed by those conducting pharmacovigilance to assist in identifying signals to be explored further, usually in controlled epidemiologic studies. It is widely recognised that, because of inconsistencies in adverse event reporting, disproportionality analyses are poor surrogates for controlled epidemiologic studies. Thus, disproportionality measures seem roughly on a par with qualitative examination of individual case reports, which is considered much less compelling than controlled epidemiologic studies. Nevertheless, regulatory action is sometimes prompted by spontaneous reports in the absence of data from controlled studies. In this issue, Rothman et al. point out that if (emphasis added) a spontaneous reporting database can be viewed as the source of case and control data for a case-control study, then the ROR is an unbiased estimate of the relative risk. While this is theoretically true, the assumptions under which disproportionality measures approximate relative risks are unverifiable, and likely to be false. For example, disproportionality measures assume that incompleteness in reporting will be constant for all event types for a particular drug. However, non-biologic factors such as time since introduction and regulatory attention can spur reporting of specific events for specific drugs, which violates this assumption. Further, the advice of Rothman et al. to ‘exclude from the control series those categories of adverse events in the database for which the occurrence or reporting of the adverse event is suspected to be related to the medication being evaluated’ is puzzling, since clinicians report adverse drug events precisely because they suspect that the event may be related to the drug. Therefore, to follow this advice, one would need to exclude all events in the control series, rendering the ROR incalculable. Thus, while one can analyse collections of uncontrolled anecdotes, and doing so may be useful for raising hypotheses, interpreting the results as if they arose from a controlled epidemiologic study is fallacious. This is not to disparage the use of disproportionality measures to identify signals in the context of pharmacovigilance, but merely their interpretation as epidemiologic effect measures. This issue also contains a report of a disproportionality analysis of Bell’s palsy in association with inactivated influenza vaccine. Given the weaknesses of disproportionality analyses of spontaneous reporting databases, should they be published in scholarly journals at all? Or might inclusion of features usually