Abstract:Background: Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians' records, it has yet to be widely applied to pharmaceutical care records. Objective: In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients' diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians' records. Methods: MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score. Results: The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data. Conclusions: MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.

Neural Disease Named Entity Extraction with Character-based BiLSTM+CRF in Japanese Medical Text

Character-level neural network for biomedical named entity recognition

Extracting Symptom Names and Disease-Symptom Relationships from Web Texts Using a Multi-Column Convolutional Neural Network

Long short-term memory RNN for biomedical named entity recognition

Named Entity Extraction for Chinese Electronic Medical Records.

Medical Named Entity Recognition Fusing Part-of-Speech and Stroke Features

Using the Natural Language Processing System Medical Named Entity Recognition-Japanese to Analyze Pharmaceutical Care Records: Natural Language Processing Analysis

Bi-LSTM-CRF Network for Clinical Event Extraction With Medical Knowledge Features

Improving Multi-Word Entity Recognition for Biomedical Texts

Named Entity Recognition of Medical Examination Reports Based on BiLSTM+CRF Model

Biomedical named entity recognition using deep neural networks with contextual information

Research on medical named entity recognition based on DB-MA-BiLSTM-CRF

Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition

Unified neural architecture for drug, disease, and clinical entity recognition

Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT

End-to-End Text Classification via Image-based Embedding using Character-level Networks

Named Entity Recognition in Chinese Electronic Medical Records Based on CRF.

Named Entity Recognition with Bidirectional LSTM-CNNs

Morphological Analysis of Japanese Hiragana Sentences using the BI-LSTM CRF Model

Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models

Medical Knowledge Attention Enhanced Neural Model for Named Entity Recognition in Chinese EMR.