Abstract:Background: Previous state-of-the-art systems on Drug Name Recognition (DNR) and Clinical Concept Extraction (CCE) have focused on a combination of text "feature engineering" and conventional machine learning algorithms such as conditional random fields and support vector machines. However, developing good features is inherently heavily time-consuming. Conversely, more modern machine learning approaches such as recurrent neural networks (RNNs) have proved capable of automatically learning effective features from either random assignments or automated word "embeddings". Objectives: (i) To create a highly accurate DNR and CCE system that avoids conventional, time-consuming feature engineering. (ii) To create richer, more specialized word embeddings by using health domain datasets such as MIMIC-III. (iii) To evaluate our systems over three contemporary datasets. Methods: Two deep learning methods, namely the Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model is set as the baseline to compare the deep learning systems to a traditional machine learning approach. The same features are used for all the models. Results: We have obtained the best results with the Bidirectional LSTM-CRF model, which has outperformed all previously proposed systems. The specialized embeddings have helped to cover unusual words in DrugBank and MedLine, but not in the i2b2/VA dataset. Conclusions: We present a state-of-the-art system for DNR and CCE. Automated word embeddings has allowed us to avoid costly feature engineering and achieve higher accuracy. Nevertheless, the embeddings need to be retrained over datasets that are adequate for the domain, in order to adequately cover the domain-specific vocabulary.

BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection

Learning Term Embeddings for Hypernymy Identification.

Word Embedding Projection Models for Hypernymy Relation Prediction

A Two-channel Model for Relation Extraction Using Multiple Trained Word Embeddings.

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Transductive Non-linear Learning for Chinese Hypernym Prediction

Spherere: Distinguishing Lexical Relations With Hyperspherical Relation Embeddings

Hypernym Relation Classification Based on Word Pattern

SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings

KEML: A Knowledge-Enriched Meta-Learning Framework for Lexical Relation Classification

Towards Bridged Vision and Language: Learning Cross-modal Knowledge Representation for Relation Extraction

Learning Bilingual Sentiment-Specific Word Embeddings without Cross-Lingual Supervision

Term Definitions Help Hypernymy Detection

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Improving Hypernymy Prediction Via Taxonomy Enhanced Adversarial Learning

Dual Supervision Framework for Relation Extraction with Distant Supervision and Human Annotation

BERE: An accurate distantly supervised biomedical entity relation extraction network

Bias Modeling for Distantly Supervised Relation Extraction

A Family of Fuzzy Orthogonal Projection Models for Monolingual and Cross-lingual Hypernymy Prediction

REKER: Relation Extraction with Knowledge of Entity and Relation.

Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition