Abstract:In recent years, with the growing amount of biomedical documents, coupled with advancement in natural language processing algorithms, the research on biomedical named entity recognition (BioNER) has increased exponentially. However, BioNER research is challenging as NER in the biomedical domain are: (i) often restricted due to limited amount of training data, (ii) an entity can refer to multiple types and concepts depending on its context and, (iii) heavy reliance on acronyms that are sub-domain specific. Existing BioNER approaches often neglect these issues and directly adopt the state-of-the-art (SOTA) models trained in general corpora, which often yields unsatisfactory results. We propose biomedical ALBERT (A Lite Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) - bioALBERT - an effective domain-specific pre-trained language model trained on a huge biomedical corpus designed to capture biomedical context-dependent NER. We adopted a self-supervised loss function used in ALBERT that targets modelling inter-sentence coherence to better learn context-dependent representations and incorporated parameter reduction strategies to minimise memory usage and enhance the training time in BioNER. In our experiments, BioALBERT outperformed comparative SOTA BioNER models on 8 biomedical NER benchmark datasets with 4 different entity types. The performance is increased for; (i) disease type corpora by 7.47% (NCBI- disease) and 10.63% (BC5CDR-disease); (ii) drug-chem type corpora by 4.61 % (BC5CDR-Chem) and 3.89% (BC4CHEMD); (iii) gene-protein type corpora by 12.25% (BC2GM) and 6.42% (JNLPBA); and (iv) species type corpora by 6.19% (LINNAEUS) and 23.71 % (Species-800) is observed which leads to a state-of-the-art results. The performance of a proposed model on four different biomedical entity types shows that our model is robust and generalisable in recognising biomedical entities in text.

Entity-Assisted Language Models for Identifying Check-worthy Sentences

Deciphering Political Entity Sentiment in News with Large Language Models: Zero-Shot and Few-Shot Strategies

An in-depth analysis of pre-trained embeddings for entity resolution

Double-Checker: Large Language Model As a Checker for Few-shot Named Entity Recognition

Entity Matching using Large Language Models

Toward the Adoption of Explainable Pre-Trained Large Language Models for Classifying Human-Written and AI-Generated Sentences

CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks

Learning Cross-Context Entity Representations from Text

A BERT based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts

Unveiling Large Language Models Generated Texts: A Multi-Level Fine-Grained Detection Framework

Pre-trained Embeddings for Entity Resolution: An Experimental Analysis [Experiment, Analysis & Benchmark]

Unlocking the Power of Large Language Models for Entity Alignment

Learning to Extract Structured Entities Using Language Models

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

Speech-to-Text Adapter and Speech-to-Entity Retriever Augmented LLMs for Speech Understanding

Improving Legal Entity Recognition Using a Hybrid Transformer Model and Semantic Filtering Approach

Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing

What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

VANER: Leveraging Large Language Model for Versatile and Adaptive Biomedical Named Entity Recognition

Structured Entity Extraction Using Large Language Models

Entities as Experts: Sparse Memory Access with Entity Supervision