Abstract:Motivation: Recognition of biomedical entities from scientific text is a critical component of natural language processing and automated information extraction platforms. Modern named entity recognition approaches rely heavily on supervised machine learning techniques, which are critically dependent on annotated training corpora. These approaches have been shown to perform well when trained and tested on the same source. However, in such scenario, the performance and evaluation of these models may be optimistic, as such models may not necessarily generalize to independent corpora, resulting in potential non-optimal entity recognition for large-scale tagging of widely diverse articles in databases such as PubMed. Results: Here we aggregated published corpora for the recognition of biomolecular entities (such as genes, RNA, proteins, variants, drugs and metabolites), identified entity class overlap and performed leave-corpus-out cross validation strategy to test the efficiency of existing models. We demonstrate that accuracies of models trained on individual corpora decrease substantially for recognition of the same biomolecular entity classes in independent corpora. This behavior is possibly due to limited generalizability of entity-class-related features captured by individual corpora (model 'overtraining') which we investigated further at the orthographic level, as well as potential annotation standard differences. We show that the combined use of multi-source training corpora results in overall more generalizable models for named entity recognition, while achieving comparable individual performance. By performing learning-curve-based power analysis we further identified that performance is often not limited by the quantity of the annotated data. Availability and implementation: Compiled primary and secondary sources of the aggregated corpora are available on: https://github.com/dterg/biomedical_corpora/wiki and https://bitbucket.org/iAnalytica/bioner. Supplementary information: Supplementary data are available at Bioinformatics online.

Identifying named entities in academic biographies with supervised learning

Exploring named‐entity recognition techniques for academic books

Automatically Building Large-Scale Named Entity Recognition Corpora from Chinese Wikipedia

Exploiting and assessing multi-source data for supervised biomedical named entity recognition

Assisted neuroscience knowledge extraction via machine learning applied to neural reconstruction metadata on NeuroMorpho.Org

Development of Biomedical Corpus Enlargement Platform Using BERT for Bio-entity Recognition

Online biomedical named entities recognition by data and knowledge-driven model

Social Network Extraction of Academic Researchers

Name disambiguation using many-to-one features

Intent Detection and Entity Extraction from BioMedical Literature

A Brief History of Named Entity Recognition

AI-assisted Knowledge Discovery in Biomedical Literature to Support Decision-making in Precision Oncology

Towards Bootstrapping Biomedical Named Entity Recognition Using Reinforcement Learning

Robust Named Entity Recognition in Idiosyncratic Domains

BioMNER: A Dataset for Biomedical Method Entity Recognition

Large-scale entity representation learning for biomedical relationship extraction

Extracting Academic Information from Conference Web Pages

NanoNER: Named Entity Recognition for nanobiology using experts' knowledge and distant supervision

Named Entity Resolution in Personal Knowledge Graphs

NERO: a biomedical named-entity (recognition) ontology with a large, annotated corpus reveals meaningful associations through text embedding