Abstract:Background: Unstructured text in medical records, such as Electronic Health Records, contain an enormous amount of valuable information for research; however, it is difficult to extract and structure important information because of frequent typographical errors. Therefore, improving the quality of data with errors for text analysis is an essential task. To date, few prior studies have been conducted addressing this. Here, we propose a new methodology for extracting important information from unstructured medical texts by overcoming the typographical problem in surgical pathology records related to lung cancer. Methods: We propose a typo correction model that considers context, based on the Masked Language Model, to solve the problem of typographical errors in real-world medical data. In addition, a word dictionary was used for the typo correction model based on PubMed abstracts. After refining the data through typo correction, fine tuning was performed on pre-trained BERT model. Next, deep learning-based Named Entity Recognition (NER) was performed. By solving the quality problem of medical data, we sought to improve the accuracy of information extraction in unstructured text data. Results: We compared the performance of the proposed typo correction model based on contextual information with an existing SymSpell model. We confirmed that our proposed model outperformed the existing model in a typographical correction task. The F1-score of the model improved by approximately 5% and 9% when compared with the model without contextual information in the NCBI-disease and surgical pathology record datasets, respectively. In addition, the F1-score of NER after typo correction increased by 2% in the NCBI-disease dataset. There was a significant performance difference of approximately 25% between the before and after typo correction in the Surgical pathology record dataset. This confirmed that typos influenced the information extraction of the unstructured text. Conclusion: We verified that typographical errors in unstructured text negatively affect the performance of natural language processing tasks. The proposed method of a typo correction model outperformed the existing SymSpell model. This study shows that the proposed model is robust and can be applied in real-world environments by focusing on the typos that cause difficulties in analyzing unstructured medical text.

Automatic Error Checking and Correction of Electronic Medical Records.

Clinical Named Entity Recognition from Chinese Electronic Medical Records Based on Deep Learning Pretraining

MLM-based typographical error correction of unstructured medical texts for named entity recognition

Automatic Conversion of Electronic Medical Record Text for OpenEHR Based on Semantic Analysis

Improving the Named Entity Recognition of Chinese Electronic Medical Records by Combining Domain Dictionary and Rules

An Automated Data Verification Approach for Improving Data Quality in a Clinical Registry.

Medical entity recognition and knowledge map relationship analysis of Chinese EMRs based on improved BiLSTM-CRF

Integrating Knowledge Retrieval and Large Language Models for Clinical Report Correction

Recognition of Chinese Electronic Medical Records for Rehabilitation Robots: Information Fusion Classification Strategy

The Development of Personalized Writing Assistant for Electronic Discharge Summaries Based on Named Entity Recognition

Named Entity Recognition in Chinese Electronic Medical Records Based on CRF.

A weakly supervised method for named entity recognition of Chinese electronic medical records

Construction, evaluation, and application of an electronic medical record corpus for cerebral palsy rehabilitation

MedAutoCorrect: Image-Conditioned Autocorrection in Medical Reporting

Named Entity Recognition of Chinese Electronic Medical Records Based on Cascaded Conditional Random Field

A comprehensive study of named entity recognition in Chinese clinical text

Black-Box Segmentation of Electronic Medical Records

An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records

A taxonomy for advancing systematic error analysis in multi-site electronic health record-based clinical concept extraction

Improving Chinese Electronic Medical Record Retrieval by Field Weight Assignment, Negation Detection, and Re-ranking

Chinese medical entity recognition based on the dual-branch TENER model