Abstract:Background: The evidence-based medicine paradigm requires the ability to aggregate and compare outcomes of interventions across different trials. This can be facilitated and partially automatized by information extraction systems. In order to support the development of systems that can extract information from published clinical trials at a fine-grained and comprehensive level to populate a knowledge base, we present a richly annotated corpus at two levels. At the first level, entities that describe components of the PICO elements (e.g., population's age and pre-conditions, dosage of a treatment, etc.) are annotated. The second level comprises schema-level (i.e., slot-filling templates) annotations corresponding to complex PICO elements and other concepts related to a clinical trial (e.g. the relation between an intervention and an arm, the relation between an outcome and an intervention, etc.). Results: The final corpus includes 211 annotated clinical trial abstracts with substantial agreement between annotators at the entity and scheme level. The mean Kappa value for the glaucoma and T2DM corpora was 0.74 and 0.68, respectively, for single entities. The micro-averaged F1 score to measure inter-annotator agreement for complex entities (i.e. slot-filling templates) was 0.81.The BERT-base baseline method for entity recognition achieved average micro- F1 scores of 0.76 for glaucoma and 0.77 for diabetes with exact matching. Conclusions: In this work, we have created a corpus that goes beyond the existing clinical trial corpora, since it is annotated in a schematic way that represents the classes and properties defined in an ontology. Although the corpus is small, it has fine-grained annotations and could be used to fine-tune pre-trained machine learning models and transformers to the specific task of extracting information about clinical trial abstracts.For future work, we will use the corpus for training information extraction systems that extract single entities, and predict template slot-fillers (i.e., class data/object properties) to populate a knowledge base that relies on the C-TrO ontology for the description of clinical trials. The resulting corpus and the code to measure inter-annotation agreement and the baseline method are publicly available at https://zenodo.org/record/6365890.

PICO entity extraction for preclinical animal literature

Advancing PICO Element Detection in Biomedical Text via Deep Neural Networks

Towards precise PICO extraction from abstracts of randomized controlled trials using a section-specific learning approach

Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study

AlpaPICO: Extraction of PICO frames from clinical trial documents using LLMs

Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification

A span-based model for extracting overlapping PICO entities from randomized controlled trial publications

Comparing generative and extractive approaches to information extraction from abstracts describing randomized clinical trials

Sent2Span: Span Detection for PICO Extraction in the Biomedical Text without Span Annotations

A Novel Framework to Expedite Systematic Reviews by Automatically Building Information Extraction Training Corpora

PICO to PICOS: Weak Supervision to Extend Datasets with New Labels

A Scoping Review of Adopted Information Extraction Methods for RCTs

Extracting PICO elements from RCT abstracts using 1-2gram analysis and multitask classification

An annotated corpus of clinical trial publications supporting schema-based relational information extraction

The use of ‘PICO for synthesis’ and methods for synthesis without meta-analysis: protocol for a survey of current practice in systematic reviews of health interventions

Rethinking PICO in the Machine Learning Era: ML-PICO

Aceso: PICO-Guided Evidence Summarization on Medical Literature

Automating data extraction in systematic reviews: a systematic review

Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations

Research Questions with PICO: A Universal Mnemonic