Abstract:Background Medical entity recognition is a key technology that supports the development of smart medicine. Existing methods on English medical entity recognition have undergone great development, but their progress in the Chinese language has been slow. Because of limitations due to the complexity of the Chinese language and annotated corpora, these methods are based on simple neural networks, which cannot effectively extract the deep semantic representations of electronic medical records (EMRs) and be used on the scarce medical corpora. We thus developed a new Chinese EMR (CEMR) dataset with six types of entities and proposed a multi-level representation learning model based on Bidirectional Encoder Representation from Transformers (BERT) for Chinese medical entity recognition. Objective This study aimed to improve the performance of the language model by having it learn multi-level representation and recognize Chinese medical entities. Methods In this paper, the pretraining language representation model was investigated; utilizing information not only from the final layer but from intermediate layers was found to affect the performance of the Chinese medical entity recognition task. Therefore, we proposed a multi-level representation learning model for entity recognition in Chinese EMRs. Specifically, we first used the BERT language model to extract semantic representations. Then, the multi-head attention mechanism was leveraged to automatically extract deeper semantic information from each layer. Finally, semantic representations from multi-level representation extraction were utilized as the final semantic context embedding for each token and we used softmax to predict the entity tags. Results The best F1 score reached by the experiment was 82.11% when using the CEMR dataset, and the F1 score when using the CCKS (China Conference on Knowledge Graph and Semantic Computing) 2018 benchmark dataset further increased to 83.18%. Various comparative experiments showed that our proposed method outperforms methods from previous work and performs as a new state-of-the-art method. Conclusions The multi-level representation learning model is proposed as a method to perform the Chinese EMRs entity recognition task. Experiments on two clinical datasets demonstrate the usefulness of using the multi-head attention mechanism to extract multi-level representation as part of the language model.

Chinese Medical Named Entity Recognition Based on Multi-word Segmentation and Multi-layer BILSTM

Clinical Named Entity Recognition from Chinese Electronic Medical Records Based on Deep Learning Pretraining

Medical named entity recognition of Chinese electronic medical records based on stacked Bidirectional Long Short-Term Memory

Chinese Medical Entity Recognition Model Based on Character and Word Vector Fusion

Medical Named Entity Recognition Based on Multi Feature Fusion of BERT

Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree

A BERT-BiLSTM-CRF Model for Chinese Electronic Medical Records Named Entity Recognition

Improving the Named Entity Recognition of Chinese Electronic Medical Records by Combining Domain Dictionary and Rules

Multi-Level Representation Learning for Chinese Medical Entity Recognition: Model Development and Validation

A Named Entity Recognition Approach for Electronic Medical Records Using BERT Semantic Enhancement and BiLSTM

Biomedical named entity recognition based on fusion multi-features embedding

Chinese Medical Record Entity Recognition Based on Lexicon and Self-attention

A Bidirectional LSTM and Conditional Random Fields Approach to Medical Named Entity Recognition.

Medical entity recognition and knowledge map relationship analysis of Chinese EMRs based on improved BiLSTM-CRF

A Hybrid Model Based on Deep Convolutional Network for Medical Named Entity Recognition

A BERT-Based Named Entity Recognition in Chinese Electronic Medical Record

Chinese Medical Nested Named Entity Recognition Model Based on Feature Fusion and Bidirectional Lattice Embedding Graph

A Hybrid Approach for Named Entity Recognition in Chinese Electronic Medical Record

Incorporating lexicon and character glyph and morphological features into BiLSTM-CRF for Chinese medical NER

A Neural Framework for Chinese Medical Named Entity Recognition.

Medical Named Entity Recognition Fusing Part-of-Speech and Stroke Features