Abstract:Many research problems involving medical texts have limited amounts of annotated data available (e.g., expressions of rare diseases). Traditional supervised machine learning algorithms, particularly those based on deep neural networks, require large volumes of annotated data, and they underperform when only small amounts of labeled data are available. Few-shot learning (FSL) is a category of machine learning models that are designed with the intent of solving problems that have small annotated datasets available. However, there is no current study that compares the performances of FSL models with traditional models (e.g., conditional random fields) for medical text at different training set sizes. In this paper, we attempted to fill this gap in research by comparing multiple FSL models with traditional models for the task of named entity recognition (NER) from medical texts. Using five health-related annotated NER datasets, we benchmarked three traditional NER models based on BERT-BERT-Linear Classifier (BLC), BERT-CRF (BC) and SANER; and three FSL NER models-StructShot & NNShot, Few-Shot Slot Tagging (FS-ST) and ProtoNER. Our benchmarking results show that almost all models, whether traditional or FSL, achieve significantly lower performances compared to the state-of-the-art with small amounts of training data. For the NER experiments we executed, the F1-scores were very low with small training sets, typically below 30%. FSL models that were reported to perform well on non-medical texts significantly underperformed, compared to their reported best, on medical texts. Our experiments also suggest that FSL methods tend to perform worse on data sets from noisy sources of medical texts, such as social media (which includes misspellings and colloquial expressions), compared to less noisy sources such as medical literature. Our experiments demonstrate that the current state-of-the-art FSL systems are not yet suitable for effective NER in medical natural language processing tasks, and further research needs to be carried out to improve their performances. Creation of specialized, standardized datasets replicating real-world scenarios may help to move this category of methods forward.

Few-shot Learning for Named Entity Recognition in Medical Text

A comparison of few-shot and traditional named entity recognition models for medical text

How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain

MedNER: Enhanced Named Entity Recognition in Medical Corpus via Optimized Balanced and Deep Active Learning

Demonstration-based learning for few-shot biomedical named entity recognition under machine reading comprehension

Few-Shot Named Entity Recognition Via Meta-Learning (extended Abstract).

Few-shot biomedical named entity recognition via knowledge-guided instance generation and prompt contrastive learning

Comparing a Large Language Model with Previous Deep Learning Models on Named Entity Recognition of Adverse Drug Events

Biomedical Named Entity Recognition at Scale

LLMs in Biomedicine: A study on clinical Named Entity Recognition

Few-shot learning for medical text: A review of advances, trends, and opportunities

Few-shot learning for medical text: A systematic review

Few-shot Named Entity Recognition: definition, taxonomy and research directions

Deep learning with word embeddings improves biomedical named entity recognition

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-shot Named Entity Recognition

CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition

A pre-training and self-training approach for biomedical named entity recognition

Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognition

An improved data augmentation approach and its application in medical named entity recognition