Abstract:A massive number of biological entities, such as genes and mutations, are mentioned in the biomedical literature. The capturing of the semantic relatedness of biological entities is vital to many biological applications, such as protein-protein interaction prediction and literature-based discovery. Concept embeddings-which involve the learning of vector representations of concepts using machine learning models-have been employed to capture the semantics of concepts. To develop concept embeddings, named-entity recognition (NER) tools are first used to identify and normalize concepts from the literature, and then different machine learning models are used to train the embeddings. Despite multiple attempts, existing biomedical concept embeddings generally suffer from suboptimal NER tools, small-scale evaluation, and limited availability. In response, we employed high-performance machine learning-based NER tools for concept recognition and trained our concept embeddings, BioConceptVec, via four different machine learning models on ~30 million PubMed abstracts. BioConceptVec covers over 400,000 biomedical concepts mentioned in the literature and is of the largest among the publicly available biomedical concept embeddings to date. To evaluate the validity and utility of BioConceptVec, we respectively performed two intrinsic evaluations (identifying related concepts based on drug-gene and gene-gene interactions) and two extrinsic evaluations (protein-protein interaction prediction and drug-drug interaction extraction), collectively using over 25 million instances from nine independent datasets (17 million instances from six intrinsic evaluation tasks and 8 million instances from three extrinsic evaluation tasks), which is, by far, the most comprehensive to our best knowledge. The intrinsic evaluation results demonstrate that BioConceptVec consistently has, by a large margin, better performance than existing concept embeddings in identifying similar and related concepts. More importantly, the extrinsic evaluation results demonstrate that using BioConceptVec with advanced deep learning models can significantly improve performance in downstream bioinformatics studies and biomedical text-mining applications. Our BioConceptVec embeddings and benchmarking datasets are publicly available at https://github.com/ncbi-nlp/BioConceptVec.

Concept-Based Embeddings for Natural Language Processing

Do Multi-Sense Embeddings Improve Natural Language Understanding?

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale

Learning Concept Embeddings for Efficient Bag-of-Concepts Densification

Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Off

Interpreting Embedding Spaces by Conceptualization

Knowledge graph embedding with concepts

ConceptX: A Framework for Latent Concept Analysis

Enhancing Semantic Word Representations by Embedding Deeper Word Relationships

Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings

Concept Representation by Learning Explicit and Implicit Concept Couplings

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Lexicon Infused Phrase Embeddings for Named Entity Resolution

Topic medical concept embedding: Multi-sense representation learning for medical concept

Refined Global Word Embeddings Based on Sentiment Concept for Sentiment Analysis

Concept2vec: Metrics for Evaluating Quality of Embeddings for Ontological Concepts

Interpretable Embedding for Ad-hoc Video Search

Discovering emergent connections in quantum physics research via dynamic word embeddings

Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency

A Language Model based Framework for New Concept Placement in Ontologies

Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding