Abstract:Background: Most current state-of-the-art models for searching the International Classification of Diseases, Tenth Revision Clinical Modification (ICD-10-CM) codes use word embedding technology to capture useful semantic properties. However, they are limited by the quality of initial word embeddings. Word embedding trained by electronic health records (EHRs) is considered the best, but the vocabulary diversity is limited by previous medical records. Thus, we require a word embedding model that maintains the vocabulary diversity of open internet databases and the medical terminology understanding of EHRs. Moreover, we need to consider the particularity of the disease classification, wherein discharge notes present only positive disease descriptions. Objective: We aimed to propose a projection word2vec model and a hybrid sampling method. In addition, we aimed to conduct a series of experiments to validate the effectiveness of these methods. Methods: We compared the projection word2vec model and traditional word2vec model using two corpora sources: English Wikipedia and PubMed journal abstracts. We used seven published datasets to measure the medical semantic understanding of the word2vec models and used these embeddings to identify the three-character-level ICD-10-CM diagnostic codes in a set of discharge notes. On the basis of embedding technology improvement, we also tried to apply the hybrid sampling method to improve accuracy. The 94,483 labeled discharge notes from the Tri-Service General Hospital of Taipei, Taiwan, from June 1, 2015, to June 30, 2017, were used. To evaluate the model performance, 24,762 discharge notes from July 1, 2017, to December 31, 2017, from the same hospital were used. Moreover, 74,324 additional discharge notes collected from seven other hospitals were tested. The F-measure, which is the major global measure of effectiveness, was adopted. Results: In medical semantic understanding, the original EHR embeddings and PubMed embeddings exhibited superior performance to the original Wikipedia embeddings. After projection training technology was applied, the projection Wikipedia embeddings exhibited an obvious improvement but did not reach the level of original EHR embeddings or PubMed embeddings. In the subsequent ICD-10-CM coding experiment, the model that used both projection PubMed and Wikipedia embeddings had the highest testing mean F-measure (0.7362 and 0.6693 in Tri-Service General Hospital and the seven other hospitals, respectively). Moreover, the hybrid sampling method was found to improve the model performance (F-measure=0.7371/0.6698). Conclusions: The word embeddings trained using EHR and PubMed could understand medical semantics better, and the proposed projection word2vec model improved the ability of medical semantics extraction in Wikipedia embeddings. Although the improvement from the projection word2vec model in the real ICD-10-CM coding task was not substantial, the models could effectively handle emerging diseases. The proposed hybrid sampling method enables the model to behave like a human expert.

Distributed representations of diseases based on co-occurrence relationship

Accelerating discoveries in medicine using distributed vector representations of words

Heterogeneous Network Embedding Enabling Accurate Disease Association Predictions

Disease2Vec: Representing Alzheimer's Progression via Disease Embedding Tree

Leveraging graph-based hierarchical medical entity embedding for healthcare applications

Voxel2vec: A Natural Language Processing Approach to Learning Distributed Representations for Scientific Data.

Identifying disease-related microbes based on multi-scale variational graph autoencoder embedding Wasserstein distance

Disease2Vec: Encoding Alzheimer's progression via disease embedding tree

Voxel2vec: A Natural Language Processing Approach to Learning Distributed Representations for Scientific Data

DapBCH: a disease association prediction model Based on Cross-species and Heterogeneous graph embedding

Snomed2Vec: Random Walk and Poincaré Embeddings of a Clinical Knowledge Base for Healthcare Analytics

Representation Learning of EHR Data via Graph-Based Medical Entity Embedding

HGECDA: A Heterogeneous Graph Embedding Model for CircRNA-Disease Association Prediction

Discovering Regularities from Traditional Chinese Medicine Prescriptions via Bipartite Embedding Model

DAEMDA: A Method with Dual-Channel Attention Encoding for miRNA-Disease Association Prediction

Distributed learning from multiple EHR databases: Contextual embedding models for medical events

High-quality gene/disease embedding in a multi-relational heterogeneous graph after a joint matrix/tensor decomposition

Representation Learning of Human Disease Mechanisms for a Foundation Model in Rare and Common Diseases

Combined embedding model for MiRNA-disease association prediction

Projection Word Embedding Model With Hybrid Sampling Training for Classifying ICD-10-CM Codes: Longitudinal Observational Study

Recognition of Hotspot Words for Disease Symptoms Incorporating Contextual Weight and Co-Occurrence Degree