EasyNER: A Customizable Easy-to-Use Pipeline for Deep Learning- and Dictionary-based Named Entity Recognition from Medical Text

Rafsan Ahmed,Petter Berntsson,Alexander Skafte,Salma Kazemi Rashed,Marcus Klang,Adam Barvesten,Ola Olde,William Lindholm,Antton Lamarca Arrizabalaga,Pierre Nugues,Sonja Aits
2024-03-07
Abstract:Background Medical research generates millions of publications and it is a great challenge for researchers to utilize this information in full since its scale and complexity greatly surpasses human reading capabilities. Automated text mining can help extract and connect information spread across this large body of literature but this technology is not easily accessible to life scientists. Results Here, we developed an easy-to-use end-to-end pipeline for deep learning- and dictionary-based named entity recognition (NER) of typical entities found in medical research articles, including diseases, cells, chemicals, genes/proteins, and species. The pipeline can access and process large medical research article collections (PubMed, CORD-19) or raw text and incorporates a series of deep learning models fine-tuned on the HUNER corpora collection. In addition, the pipeline can perform dictionary-based NER related to COVID-19 and other medical topics. Users can also load their own NER models and dictionaries to include additional entities. The output consists of publication-ready ranked lists and graphs of detected entities and files containing the annotated texts. An associated script allows rapid inspection of the results for specific entities of interest. As model use cases, the pipeline was deployed on two collections of autophagy-related abstracts from PubMed and on the CORD19 dataset, a collection of 764 398 research article abstracts related to COVID-19. Conclusions The NER pipeline we present is applicable in a variety of medical research settings and makes customizable text mining accessible to life scientists.
Quantitative Methods,Computation and Language
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve This paper aims to address the issue of the vast and complex volume of medical research literature, which makes it difficult for researchers to fully utilize this information. Specifically, the paper develops an easy-to-use end-to-end Named Entity Recognition (NER) pipeline for identifying common entity types from medical texts, including diseases, cells, chemicals, genes/proteins, and species. This pipeline is capable of handling large-scale collections of medical literature (such as PubMed and CORD-19 databases) as well as raw texts, and it incorporates a series of deep learning models fine-tuned on the HUNER corpus. Additionally, the pipeline supports dictionary-based NER, particularly for entities related to COVID-19 and other medical topics. The main contributions of the paper are: 1. **Development of an easy-to-use NER pipeline**: This pipeline allows life scientists to perform customized text mining without a deep understanding of complex natural language processing techniques. 2. **Support for multiple entity types**: The pipeline can identify various common entity types in medical research, including cells, chemicals, diseases, genes/proteins, and species. 3. **Compatibility with multiple input formats**: The pipeline can process PubMed ID lists, batch files from the PubMed database, CORD-19 metadata files, or free text files as input. 4. **Provision of rich output formats**: The pipeline outputs include ranked lists of identified entities, charts, and structured annotation files, facilitating subsequent analysis. With these features, the pipeline significantly enhances the ability of life science researchers to extract useful information from a large volume of medical literature.