Abstract:Abstract Motivation Large Language Models (LLMs) have the potential to revolutionize the field of Natural Language Processing (NLP), excelling not only in text generation and reasoning tasks but also in their ability for zero/few-shot learning, swiftly adapting to new tasks with minimal fine-tuning. LLMs have also demonstrated great promise in biomedical and healthcare applications. However, when it comes to Named Entity Recognition (NER), particularly within the biomedical domain, LLMs fall short of the effectiveness exhibited by fine-tuned domain-specific models. One key reason is that NER is typically conceptualized as a sequence labeling task, whereas LLMs are optimized for text generation and reasoning tasks. Results We developed an instruction-based learning paradigm that transforms biomedical NER from a sequence labeling task into a generation task. This paradigm is end-to-end and streamlines the training and evaluation process by automatically repurposing pre-existing biomedical NER datasets. We further developed BioNER-LLaMA using the proposed paradigm with LLaMA-7B as the foundational LLM. We conducted extensive testing on BioNER-LLaMA across three widely recognized biomedical NER datasets, consisting of entities related to diseases, chemicals, and genes. The results revealed that BioNER-LLaMA consistently achieved higher F1-scores ranging from 5% to 30% compared to the few-shot learning capabilities of GPT-4 on datasets with different biomedical entities. We show that a general-domain LLM can match the performance of rigorously fine-tuned PubMedBERT models and PMC-LLaMA, biomedical-specific language model. Our findings underscore the potential of our proposed paradigm in developing general-domain LLMs that can rival SOTA performances in multi-task, multi-domain scenarios in biomedical and health applications. Availability Datasets and other resources are available at https://github.com/BIDS-Xu-Lab/BioNER-LLaMA. Supplementary information Supplementary data are available at Bioinformatics online.

A tool for mapping medical narratives into medical ontologies in low resource settings: A case study for German

GERNERMED: An open German medical NER model

German Medical Named Entity Recognition Model and Data Set Creation Using Machine Translation and Word Alignment: Algorithm Development and Validation

LLMs in Biomedicine: A study on clinical Named Entity Recognition

GERNERMED++: Transfer Learning in German Medical NLP

Zero-Shot LLMs for Named Entity Recognition: Targeting Cardiac Function Indicators in German Clinical Texts

GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models

Large Language Models Struggle in Token-Level Clinical Named Entity Recognition

GERNERMED++: Semantic annotation in German medical NLP through transfer-learning, translation and word alignment

Annotated dataset creation through large language models for non-english medical NLP

Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding

Embedding Transfer for Low-Resource Medical Named Entity Recognition: A Case Study on Patient Mobility

Improving the Quality of Unstructured Cancer Data Using Large Language Models: A German Oncological Case Study

Information extraction from German radiological reports for general clinical text and language understanding

Annotated Dataset Creation through General Purpose Language Models for non-English Medical NLP

CORAL: Expert-Curated medical Oncology Reports to Advance Language Model Inference

Viability of Open Large Language Models for Clinical Documentation in German Health Care: Real-World Model Evaluation Study

Mapping SNOMED CT Codes to Semi-Structured Texts via an NLP Pipeline

Advancing entity recognition in biomedicine via instruction tuning of large language models

Multilingual Medical Question Answering and Information Retrieval for Rural Health Intelligence Access