Abstract:Although randomized controlled trials (RCTs) are the gold standard for establishing the efficacy and safety of a medical treatment, real-world evidence (RWE) generated from real-world data has been vital in postapproval monitoring and is being promoted for the regulatory process of experimental therapies. An emerging source of real-world data is electronic health records (EHRs), which contain detailed information on patient care in both structured (eg, diagnosis codes) and unstructured (eg, clinical notes and images) forms. Despite the granularity of the data available in EHRs, the critical variables required to reliably assess the relationship between a treatment and clinical outcome are challenging to extract. To address this fundamental challenge and accelerate the reliable use of EHRs for RWE, we introduce an integrated data curation and modeling pipeline consisting of 4 modules that leverage recent advances in natural language processing, computational phenotyping, and causal modeling techniques with noisy data. Module 1 consists of techniques for data harmonization. We use natural language processing to recognize clinical variables from RCT design documents and map the extracted variables to EHR features with description matching and knowledge networks. Module 2 then develops techniques for cohort construction using advanced phenotyping algorithms to both identify patients with diseases of interest and define the treatment arms. Module 3 introduces methods for variable curation, including a list of existing tools to extract baseline variables from different sources (eg, codified, free text, and medical imaging) and end points of various types (eg, death, binary, temporal, and numerical). Finally, module 4 presents validation and robust modeling methods, and we propose a strategy to create gold-standard labels for EHR variables of interest to validate data curation quality and perform subsequent causal modeling for RWE. In addition to the workflow proposed in our pipeline, we also develop a reporting guideline for RWE that covers the necessary information to facilitate transparent reporting and reproducibility of results. Moreover, our pipeline is highly data driven, enhancing study data with a rich variety of publicly available information and knowledge sources. We also showcase our pipeline and provide guidance on the deployment of relevant tools by revisiting the emulation of the Clinical Outcomes of Surgical Therapy Study Group Trial on laparoscopy-assisted colectomy versus open colectomy in patients with early-stage colon cancer. We also draw on existing literature on EHR emulation of RCTs together with our own studies with the Mass General Brigham EHR.

Real-world data to support post-market safety and performance of embolization coils: evidence generation from a medical device manufacturer and data institute partnership

Real-world data to support post-market safety and performance of embolization coils: evidence generation from a medical device manufacturer and data institute partnership

Lessons on the use of real-world data in medical device research: findings from the National Evaluation System for Health Technology Test-Cases

Use of Real-World Evidence for Regulatory Approval and Coverage of Medical Devices: A Landscape Assessment

Real-World Evidence: A Primer

Utilization of real-world evidence in clinical research of medical devices

Advancing the Real-World Evidence for Medical Devices through Coordinated Registry Networks

Reality check: Real-world evidence to support therapeutic development in hematologic malignancies

Real-world evidence to support regulatory submissions: A landscape review and assessment of use cases

Use of Real‐World Evidence to Drive Drug Development Strategy and Inform Clinical Trial Design

Leveraging Real-World Data in Safety Signal Assessment

Generate Analysis-Ready Data for Real-world Evidence: Tutorial for Harnessing Electronic Health Records With Advanced Informatic Technologies

Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation

ISPOR, the FDA, and the Evolving Regulatory Science of Medical Device Products

Real-World Evidence Should Be Used in Regulatory Decisions About New Pharmaceutical and Medical Device Products for Diabetes

Marketing Authorization Applications Made to the European Medicines Agency in 2018–2019: What was the Contribution of Real‐World Evidence?

Medical device surveillance with electronic health records

How real can we get in generating real world evidence? Exploring the opportunities of routinely collected administrative data for evaluation of medical devices

Use of real‐world evidence in regulatory decisions for rare diseases in the United States—Current status and future directions

From real world data to real world evidence to improve outcomes in neuro-ophthalmology