Abstract:Background Study protocols define the population(s) of interest through a set of inclusion and exclusion criteria -often in the form of a textual description. In haematology, these criteria can become very complex and are difficult to translate into consistent and robust computer interpretable cohort definitions. Addressing this interoperability challenge has the potential to vastly accelerate the evaluation of populations of interest from a trial to a real world setting or to replicate studies. Generative AI (GenAI) and its potential to extract valuable information from textual data such as medical records or research papers has recently received a lot of attention. Building on the clinical data harmonisation that already took place across the Haematology Outcomes Network in Europe (HONEUR), the application of the latest evolutions in GenAI technology for the generation of computer interpretable cohort definitions from descriptions in study protocols has been assessed and potential limitations have been identified. Methods HONEUR is a federated data network of 21 registries and hospitals in 9 different countries, comprising 45,000 multiple myeloma patients, where the data stays local and all participating data partners have full governance (www.honeur.org). The main aim of the network is to accelerate evidence generation to expand the knowledge and understanding of patients with hematological malignancies. The pivotal step in enabling the federated analysis is the clinical data harmonization. This is based on structural and semantic harmonization to the OMOP data model (https://www.ohdsi.org/data-standardization/). For studies involving subpopulations of multiple myeloma patients, a simplified tabular structure is generated from the OMOP data. This structure contains an extensive list of baseline and line of treatment specific variables. For the conversion of the inclusion/exclusion criteria into a standard format that could be executed by the HONEUR tools, ChatGPT (https://chat.openai.com/) version 4.0 was used. The following steps were hereby followed as a set of ‘prompts’: Define the role and system prompts: explaining in plain language what the intent is and what steps will be followed.Provide information on the data structure that contains the harmonized data: the data dictionary from the tabular structure mentioned above with the variable names, the description of these and the allowed values of the variables.Providing the target structure of the output fileProvide the inclusion / exclusion criteria as “training” promptsInstruct ChatGPT on how to behave with unknown scenarios.Repeat Step 5 until the outcome file correctly represents the cohort description as provided in Step 4 The result of the above steps is a correctly formatted file that is evaluated by Nuffield Cancer Centre London through upload in the local HONEUR real-world database and evaluated to assess the validity of the cohort definition(s) Results In total 6 patients met all criteria in Nuffield Cancer Centre London (see Table 1). The system was further evaluated on more complex inclusion of cytogenetic and subsequent treatment inclusion criteria. [Table 1] Conclusions In our setting, we've found that one-shot or few-shot learning is adequate to correctly translate complex scenarios of inclusion / exclusion rules into executable cohort definitions through the use of GenAI. The upfront semantic clinical data harmonization is however a prerequisite as the tools are not yet adequately trained to detect subtle but sometimes critical nuances in the clinical criteria. The application of GenAI is promising for conversion of textual data to structured and computer interpretable information. This offers the possibility to accelerate evidence generation in a robust and reproducible way. We showed this system can work in identifying 6 patients in one center and is therefore extrapolatable to all 21 participating centers.

Cohort Builder: A Software Pipeline for Generating Patient Cohorts with Predetermined Baseline Characteristics from Medical Records and Raw Ophthalmic Imaging Data

CohortNet: Empowering Cohort Discovery for Interpretable Healthcare Analytics

Establishing the Automatic Identification of Clinical Trial Cohorts from Electronic Health Records by Matching Normalized Eligibility Criteria and Patient Clinical Characteristics

Towards a Privacy Preserving Cohort Discovery Framework for Clinical Research Networks.

Implementation of a Cohort Retrieval System for Clinical Data Repositories Using the Observational Medical Outcomes Partnership Common Data Model: Proof-of-Concept System Validation

CREATE: Cohort Retrieval Enhanced by Analysis of Text from Electronic Health Records using OMOP Common Data Model

A framework for employing longitudinally collected multicenter electronic health records to stratify heterogeneous patient populations on disease history

Toward Cohort Intelligence: A Universal Cohort Representation Learning Framework for Electronic Health Record Analysis

Longitudinal Cohorts for Harnessing the Electronic Health Record for Disease Prediction in a US Population.

Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

ODVICE: An Ontology-Driven Visual Analytic Tool for Interactive Cohort Extraction

Feasibility of cross-vendor linkage of ophthalmic images with electronic health record data: an analysis from the IRIS Registry®

An Honeur Framework for Generating Computer Understandable Cohort Definitions from Clinical Trial Protocols in Multiple Myeloma through Generative AI for Comparing with Real-World Data

[Smart eye data : Development of a foundation for medical research using Smart Data applications]

Cohort Identification for Clinical Research: Querying Federated Electronic Healthcare Records Using Controlled Vocabularies and Semantic Types.

I-ODA, Real-World Multi-modal Longitudinal Data for OphthalmicApplications

An Architecture to Define Cohorts over Medical Imaging Datasets

Scalable analysis of Big pathology image data cohorts using efficient methods and high-performance computing strategies

Leveraging Computational Pathology AI for Noninvasive Optical Imaging Analysis Without Retraining

A scalable approach for critical care data extraction and analysis in an academic medical center