Abstract:Although randomized controlled trials (RCTs) are the gold standard for establishing the efficacy and safety of a medical treatment, real-world evidence (RWE) generated from real-world data has been vital in postapproval monitoring and is being promoted for the regulatory process of experimental therapies. An emerging source of real-world data is electronic health records (EHRs), which contain detailed information on patient care in both structured (eg, diagnosis codes) and unstructured (eg, clinical notes and images) forms. Despite the granularity of the data available in EHRs, the critical variables required to reliably assess the relationship between a treatment and clinical outcome are challenging to extract. To address this fundamental challenge and accelerate the reliable use of EHRs for RWE, we introduce an integrated data curation and modeling pipeline consisting of 4 modules that leverage recent advances in natural language processing, computational phenotyping, and causal modeling techniques with noisy data. Module 1 consists of techniques for data harmonization. We use natural language processing to recognize clinical variables from RCT design documents and map the extracted variables to EHR features with description matching and knowledge networks. Module 2 then develops techniques for cohort construction using advanced phenotyping algorithms to both identify patients with diseases of interest and define the treatment arms. Module 3 introduces methods for variable curation, including a list of existing tools to extract baseline variables from different sources (eg, codified, free text, and medical imaging) and end points of various types (eg, death, binary, temporal, and numerical). Finally, module 4 presents validation and robust modeling methods, and we propose a strategy to create gold-standard labels for EHR variables of interest to validate data curation quality and perform subsequent causal modeling for RWE. In addition to the workflow proposed in our pipeline, we also develop a reporting guideline for RWE that covers the necessary information to facilitate transparent reporting and reproducibility of results. Moreover, our pipeline is highly data driven, enhancing study data with a rich variety of publicly available information and knowledge sources. We also showcase our pipeline and provide guidance on the deployment of relevant tools by revisiting the emulation of the Clinical Outcomes of Surgical Therapy Study Group Trial on laparoscopy-assisted colectomy versus open colectomy in patients with early-stage colon cancer. We also draw on existing literature on EHR emulation of RCTs together with our own studies with the Mass General Brigham EHR.

Transform: Implementing a Learning Healthcare System in Europe through Embedding Clinical Research into Clinical Practice

The Integration Challenges in Bridging Patient Care and Clinical Research in a Learning Healthcare System

Detailed clinical modelling approach to data extraction from heterogeneous data sources for clinical research.

Mobilizing Clinical Decision Support to Facilitate Knowledge Translation: a Case Study in China.

eSource for clinical trials: Implementation and evaluation of a standards-based approach in a real world trial

The TRANSFoRm project: Experience and lessons learned regarding functional and interoperability requirements to support primary care

Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory

Digitizing clinical trials

The Texas Health Resources Clinical Scholars Program: Learning healthcare system workforce development through embedded translational research

Business analysis for a sustainable, multi-stakeholder ecosystem for leveraging the Electronic Health Records for Clinical Research (EHR4CR) platform in Europe

Transforming Health Care Through a Learning Health System Approach in the Digital Era: Chronic Kidney Disease Management in China

Building to learn: Information technology innovations to enable rapid pragmatic evaluation in a learning health system

Clinical deployment environments: Five pillars of translational machine learning for health

Using a network organisational architecture to support the development of Learning Healthcare Systems

Generate Analysis-Ready Data for Real-world Evidence: Tutorial for Harnessing Electronic Health Records With Advanced Informatic Technologies

Accelerating clinical trials in the EU (ACT EU): transforming the EU clinical trials landscape

Building Capacity for Pragmatic Trials of Digital Technology in Primary Care

Interorganizational Knowledge Sharing to Establish Digital Health Learning Ecosystems: Qualitative Evaluation of a National Digital Health Transformation Program in England (Preprint)

Reverse Engineering of Digital Measures: Inviting Patients to the Conversation

Improving American Healthcare Through “Clinical Lab 2.0”

An HL7-based Information Modelling Approach for the Design of Secondary Care Research Systems