Abstract:Named entity recognition (NER) is a critical subtask in natural language processing. It is particularly valuable to gain a deeper understanding of entity boundaries and entity types when addressing the NER problem. Most previous sequential labeling models are task-specific, while recent years have witnessed the rise of generative models due to the advantage of tackling NER tasks in the encoder–decoder framework. Despite achieving promising performance, our pilot studies demonstrate that existing generative models are ineffective at detecting entity boundaries and estimating entity types. In this paper, a multiple attention framework is proposed which introduces the attention of entity-type embedding and word–word relation into the named entity recognition task. To improve the accuracy of entity-type mapping, we adopt an external knowledge base to calculate the prior entity-type distributions and then incorporate the information input to the model via the encoder's self-attention. To enhance the contextual information, we take the entity types as part of the input. Our method obtains the other attention from the hidden states of entity types and utilizes it in self- and cross-attention mechanisms in the decoder. We transform the entity boundary information in the sequence into word–word relations and extract the corresponding embedding into the cross-attention mechanism. Through word–word relation information, the method can learn and understand more entity boundary information, thereby improving its entity recognition accuracy. We performed experiments on extensive NER benchmarks, including four flat and two long entity benchmarks. Our approach significantly improves or performs similarly to the best generative NER models. The experimental results demonstrate that our method can substantially enhance the capabilities of generative NER models.

Multi-task Multi-attention Transformer for Generative Named Entity Recognition

Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition

Incorporating Entity Type-Aware and Word–Word Relation-Aware Attention in Generative Named Entity Recognition

Multi-Grained Named Entity Recognition

Hero-Gang Neural Model For Named Entity Recognition

A Unified Generative Framework for Various NER Subtasks

A Multi-Task Framework Based on Decomposition for Multimodal Named Entity Recognition

Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach

ToNER: Type-oriented Named Entity Recognition with Generative Language Model

Adversarial Multi-Task Learning for Efficient Chinese Named Entity Recognition

Joint Cross-document Information for Named Entity Recognition with Multi-task Learning

Contrastive Information Extraction with Generative Transformer

MFE-transformer: Adaptive English text named entity recognition method based on multi-feature extraction and transformer

CAT-MNER: Multimodal Named Entity Recognition with Knowledge-Refined Cross-Modal Attention

Sentence-to-Label Generation Framework for Multi-task Learning of Japanese Sentence Classification and Named Entity Recognition

Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

Learning with Joint Cross-Document Information Via Multi-Task Learning for Named Entity Recognition

DATG: Data Augmentation with Transformer-Based Generation for Low-Resource Named Entity Recognition

A Multi-task Approach for Machine Reading Comprehension Form Named Entity Recognition Tasks

Research on Named Entity Recognition Based on Multi-Task Learning and Biaffine Mechanism

Named entity recognition model based on Multi‐BiLSTM and competition mechanism