Abstract:Introduction: Several quantitative methods have been established, in pharmacovigilance, to detect signals of disproportionate reporting (SDRs) from databases containing reports of adverse drug reactions (ADRs). The signal detection algorithms (SDAs) and the source of the reporting per product vary, but it is unclear whether any algorithm can provide satisfactory performance using data with such large variance factors. Objective: Determine the appropriate SDA for Biogen's internal Global Safety Database (GSD) given the characteristics of the database including frequencies of events, data skewness, outliers, and missing information. Compare performance of standard approaches (EBGM, EB05, PRR, and ROR), well accepted by industry, to a Biogen-developed Machine Learning (ML) Regression Decision Tree (RDT) model, across several Biogen products, to determine a champion SDA. Methods: All data associated with seven marketed Biogen products were chosen and a historical subset of reported ADRs were considered. Six SDAs (five common industry disproportionality methods) and RDT were evaluated. The SDRs were calculated on training and test data composed of quarterly reporting intervals from 2004-2019. The performance measures used were sensitivity, precision, time to detect new events, and frequency of detected cases for each algorithm for each product. Outcomes in the test data are known a priori and easily compared to predicted outcomes. Validation was performed via rates of misclassification. This work solely represents Biogen's internal information, intentionally chosen to serve the performance review of its signal detection systems, and results will not necessarily be generalizable to other external sources. Results: Several algorithms performed differently among products, but no one method dominated any other. Performance was dependent on the thresholds used to define a signal according to different criteria. However, those different statistics subtly influenced the achievable performance. The relative performance of RDT and Medicines and Healthcare products Regulatory Agency (MHRA) algorithms were superior and paired across products. A reduction in precision for all methods spanning the products was present. Hence, companies evaluating signal detection approaches, search for innovative methods to minimize this effect. Conclusions: In designing signal detection systems, careful consideration should be given to the criteria that are used to define SDRs. The choice of disproportionality statistics does not affect the achievable range of signal detection performance. These choices should consider mainly ease of implementation and interpretation. The implementation of a method is specific to its accuracy. The RDT attempted to take advantage of known methods and compare results on a per-product basis. Many factors influencing ADRs may improve RDT in future efforts. In this experiment, RDT demonstrated superiority in terms of quickest time to detect and capturing of the highest number of ADRs. Next steps include expansion of data for products representing other indications and testing models in external databases to investigate generalizability of estimates when comparing SDAs.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration

The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement

The evolving role of disproportionality analysis in pharmacovigilance

Analysis of a Pharmacist-Led Adverse Drug Event Management Model for Pharmacovigilance in an Academic Medical Center Hospital in China

An Empirical Approach to Explore the Relationship Between Measures of Disproportionate Reporting and Relative Risks from Analytical Studies

Identification of Novel Signals Associated with US-FDA Approved Drugs (2013) Using Disproportionality Analysis

A characterization and disproportionality analysis of medication error related adverse events reported to the FAERS database

Investigating the Benefit-Risk Profile of Drugs: From Spontaneous Reporting Systems to Real-World Data for Pharmacovigilance

Statistical Signal Detection Algorithm in Safety Data: A Proprietary Method Compared to Industry Standard Methods

Supplementary Data S3 from Proportional Hazards Violations in Phase III Cancer Clinical Trials: A Potential Source of Trial Misinterpretation

Discovering clinical drug-drug interactions with known pharmacokinetics mechanisms using spontaneous reporting systems and electronic health records

Changing paradigms in detecting rare adverse drug reactions: from disproportionality analysis, old and new, to machine learning

Exploiting heterogeneous publicly available data sources for drug safety surveillance: computational framework and case studies

The Impact of Mandatory Reporting of Non-Serious Safety Reports to EudraVigilance on the Detection of Adverse Reactions

Web-Based Signal Detection Using Medical Forums Data in France: Comparative Analysis

Development of a multivariate prediction model to identify individual case safety reports which require clinical review

Contemporary Practice and Considerations for Real-World Data Source Identification and Feasibility Assessment

A Retrospective Observational Study of Adverse Drug Reactions (ADR) Reported to ADR Monitoring Centre from 2010 to 2020

Evaluation of a simplified pharmacovigilance tool for general practitioners: 5 years of insight

Risk Factor Considerations in Statistical Signal Detection: Using Subgroup Disproportionality to Uncover Risk Groups for Adverse Drug Reactions in VigiBase

Identifying high‐risk medications and error types in Danish patient safety database using disproportionality analysis