Character-to-Word Representation and Global Contextual Representation for Named Entity Recognition

Jun Chang,Xiaohong Han
DOI: https://doi.org/10.1007/s11063-023-11168-6
IF: 2.565
2023-01-01
Neural Processing Letters
Abstract:The essence of named entity recognition is to mine entities with specific meanings in the text, which is the basis for some downstream tasks in the field of natural language processing. Currently, deep learning-based methods have further improved the accuracy of named entity recognition, and most methods are based on word-level and character-level embeddings. However, these methods ignore the effectiveness of global context for entity recognition, so this paper proposes to use an attention mechanism to obtain comprehensive information of the same word from different contextual information. Meanwhile, character-level representations affect not only the accuracy of recognizing unseen words, but also the extraction of contextual representations. Considering this issue, we propose to extract character-to-word representations using label attention mechanism. The proposed model uses CNN-LSTM-CRF as the baseline, which is effectively integrated into the above two representation extraction methods, named CNN-CWR-LSTM-GCR-CRF. On the basis of this model, we further integrate the language model BERT. Experiments show that our model achieves the results competitive with the state-of-the-art records on CONLL-2002 Spanish dataset, CONLL-2003 and Ontonotes5.0 English datasets, respectively.
What problem does this paper attempt to address?