Abstract:Abstract Motivation Large Language Models (LLMs) have the potential to revolutionize the field of Natural Language Processing (NLP), excelling not only in text generation and reasoning tasks but also in their ability for zero/few-shot learning, swiftly adapting to new tasks with minimal fine-tuning. LLMs have also demonstrated great promise in biomedical and healthcare applications. However, when it comes to Named Entity Recognition (NER), particularly within the biomedical domain, LLMs fall short of the effectiveness exhibited by fine-tuned domain-specific models. One key reason is that NER is typically conceptualized as a sequence labeling task, whereas LLMs are optimized for text generation and reasoning tasks. Results We developed an instruction-based learning paradigm that transforms biomedical NER from a sequence labeling task into a generation task. This paradigm is end-to-end and streamlines the training and evaluation process by automatically repurposing pre-existing biomedical NER datasets. We further developed BioNER-LLaMA using the proposed paradigm with LLaMA-7B as the foundational LLM. We conducted extensive testing on BioNER-LLaMA across three widely recognized biomedical NER datasets, consisting of entities related to diseases, chemicals, and genes. The results revealed that BioNER-LLaMA consistently achieved higher F1-scores ranging from 5% to 30% compared to the few-shot learning capabilities of GPT-4 on datasets with different biomedical entities. We show that a general-domain LLM can match the performance of rigorously fine-tuned PubMedBERT models and PMC-LLaMA, biomedical-specific language model. Our findings underscore the potential of our proposed paradigm in developing general-domain LLMs that can rival SOTA performances in multi-task, multi-domain scenarios in biomedical and health applications. Availability Datasets and other resources are available at https://github.com/BIDS-Xu-Lab/BioNER-LLaMA. Supplementary information Supplementary data are available at Bioinformatics online.

A Novel Cascade Instruction Tuning Method for Biomedical NER.

Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processing

Advancing entity recognition in biomedicine via instruction tuning of large language models

BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processing

LLMs in Biomedicine: A study on clinical Named Entity Recognition

BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning

How Important is Domain Specificity in Language Models and Instruction Finetuning for Biomedical Relation Extraction?

CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model

Recognising Biomedical Names: Challenges and Solutions

Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

Fine-tuning large neural language models for biomedical natural language processing

Ensemble Transfer Learning on Augmented Domain Resources for Oncological Named Entity Recognition in Chinese Clinical Records

Instruction Mining: Instruction Data Selection for Tuning Large Language Models

Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach

GEIC: Universal and Multilingual Named Entity Recognition with Large Language Models

Structure-aware Domain Knowledge Injection for Large Language Models

UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition

A New Pipeline For Generating Instruction Dataset via RAG and Self Fine-Tuning