Abstract:Motivation: The explosive increase of biomedical literature has made information extraction an increasingly important tool for biomedical research. A fundamental task is the recognition of biomedical named entities in text (BNER) such as genes/proteins, diseases and species. Recently, a domain-independent method based on deep learning and statistical word embeddings, called long short-term memory network-conditional random field (LSTM-CRF), has been shown to outperform state-of-the-art entity-specific BNER tools. However, this method is dependent on gold-standard corpora (GSCs) consisting of hand-labeled entities, which tend to be small but highly reliable. An alternative to GSCs are silver-standard corpora (SSCs), which are generated by harmonizing the annotations made by several automatic annotation systems. SSCs typically contain more noise than GSCs but have the advantage of containing many more training examples. Ideally, these corpora could be combined to achieve the benefits of both, which is an opportunity for transfer learning. In this work, we analyze to what extent transfer learning improves upon state-of-the-art results for BNER.Results: We demonstrate that transferring a deep neural network (DNN) trained on a large, noisy SSC to a smaller, but more reliable GSC significantly improves upon state-of-the-art results for BNER. Compared to a state-of-the-art baseline evaluated on 23 GSCs covering four different entity classes, transfer learning results in an average reduction in error of approximately 11%. We found transfer learning to be especially beneficial for target datasets with a small number of labels (approximately 6000 or less).Availability and implementation: Source code for the LSTM-CRF is available at https://github.com/Franck-Dernoncourt/NeuroNER/ and links to the corpora are available at https://github.com/BaderLab/Transfer-Learning-BNER-Bioinformatics-2018/.Supplementary information: Supplementary data are available at Bioinformatics online.

Transfer Learning for Cross-Domain Sequence Tagging Tasks

Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks

Cross-domain NER under a Divide-and-Transfer Paradigm

Three Heads Are Better Than One: Improving Cross-Domain NER with Progressive Decomposed Network

Transfer learning for biomedical named entity recognition with neural networks

POISE: Efficient Cross-Domain Chinese Named Entity Recognization Via Transfer Learning

Neural Adaptation Layers for Cross-domain Named Entity Recognition

Cross-lingual, Character-Level Neural Morphological Tagging

An Instance Transfer based Approach Using Enhanced Recurrent Neural Network for Domain Named Entity Recognition

Sequence-To-Sequence Domain Adaptation Network For Robust Text Image Recognition

Multi-Task Cross-Lingual Sequence Tagging from Scratch

Transfer Learning for Sequence Labeling Using Source Model and Target Data

Transfer Learning and Deep Domain Adaptation

Cross‐domain Sequence Labelling Using Language Modelling and Parameter Generating

Cross-domain Named Entity Recognition via Graph Matching

Transfer Learning for Sequences via Learning to Collocate

Domain structure-based transfer learning for cross-domain word representation

Cross-domain NER in the data-poor scenarios for human mobility knowledge

Exploring and Predicting Transferability across NLP Tasks

A Research Toward Chinese Named Entity Recognition Based on Transfer Learning

Structure and Label Constrained Data Augmentation for Cross-domain Few-shot NER