Abstract:We present the preparation, resources, results and analysis of three tasks of the BioNLP Shared Task 2011: the main tasks on Infectious Diseases (ID) and Epigenetics and Post-translational Modifications (EPI), and the supporting task on Entity Relations (REL). The two main tasks represent extensions of the event extraction model introduced in the BioNLP Shared Task 2009 (ST'09) to two new areas of biomedical scientific literature, each motivated by the needs of specific biocuration tasks. The ID task concerns the molecular mechanisms of infection, virulence and resistance, focusing in particular on the functions of a class of signaling systems that are ubiquitous in bacteria. The EPI task is dedicated to the extraction of statements regarding chemical modifications of DNA and proteins, with particular emphasis on changes relating to the epigenetic control of gene expression. By contrast to these two application-oriented main tasks, the REL task seeks to support extraction in general by separating challenges relating to part-of relations into a subproblem that can be addressed by independent systems. Seven groups participated in each of the two main tasks and four groups in the supporting task. The participating systems indicated advances in the capability of event extraction methods and demonstrated generalization in many aspects: from abstracts to full texts, from previously considered subdomains to new ones, and from the ST'09 extraction targets to other entities and events. The highest performance achieved in the supporting task REL, 58% F-score, is broadly comparable with levels reported for other relation extraction tasks. For the ID task, the highest-performing system achieved 56% F-score, comparable to the state-of-the-art performance at the established ST'09 task. In the EPI task, the best result was 53% F-score for the full set of extraction targets and 69% F-score for a reduced set of core extraction targets, approaching a level of performance sufficient for user-facing applications. In this study, we extend on previously reported results and perform further analyses of the outputs of the participating systems. We place specific emphasis on aspects of system performance relating to real-world applicability, considering alternate evaluation metrics and performing additional manual analysis of system outputs. We further demonstrate that the strengths of extraction systems can be combined to improve on the performance achieved by any system in isolation. The manually annotated corpora, supporting resources, and evaluation tools for all tasks are available from http://www.bionlp-st.org and the tasks continue as open challenges for all interested parties.

Extracting Decision Trees from Medical Texts: an Overview of the Text2DT Track in CHIP2022

Text2MDT: Extracting Medical Decision Trees from Medical Texts

A Pattern-Based Method for Medical Entity Recognition From Chinese Diagnostic Imaging Text

The 2022 n2c2/UW shared task on extracting social determinants of health

A Study of Deep Learning Approaches for Medication and Adverse Drug Event Extraction from Clinical Text.

Advancing Chinese biomedical text mining with community challenges

Exploration of text matching methods in Chinese disease Q&A systems: A method using ensemble based on BERT and boosted tree models

Overview of the NLPCC 2022 Shared Task: Dialogue Text Analysis (DTA)

2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records

Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification

Overview of the ID, EPI and REL tasks of BioNLP Shared Task 2011

A novel deep learning approach to extract Chinese clinical entities for lung cancer screening and staging

Overview of the First Shared Task on Clinical Text Generation: RRG24 and "Discharge Me!"

Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Study

Comparative Analysis of Text Classification Approaches in Electronic Health Records

Automatic Emergency Diagnosis with Knowledge-Based Tree Decoding.

Text Mining Drug/Chemical-Protein Interactions using an Ensemble of BERT and T5 Based Models

Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data

Developing and Validating Predictive Decision Tree Models from Mining Chemical Structural Fingerprints and High–throughput Screening Data in PubChem

MIDRC-MetricTree: a decision tree-based tool for recommending performance metrics in artificial intelligence-assisted medical image analysis