RoPE-BAM: Nested Entity Recognition Based on Rotary Position Embedding and Biaffine Attention Mechanism

Jinxin Deng,Xizhong Qin,Rong Yang,Xiaoyi Lv

DOI: https://doi.org/10.1117/12.3004825

2023-01-01

Abstract:Named entity recognition (NER) involves two main types: nested NER and flat NER. The span-based approach classifies entity types by head-tail pair span representations and can handle nested and flat entities uniformly. However, the span-based approach uses a single feature and ignores the relative position information between head-tail pairs, which affects the precision of entity recognition. Therefore, we propose a nested entity recognition model that combines rotary position embedding and biaffine attention mechanism (RoPE-BAM) to improve the model performance by adding relative position features to the span representations. Concretely, we first obtain the head sequence and tail sequence representations through two feedforward networks. Then, to incorporate the relative position features, rotary position embedding is applied to both head and tail sequences. Finally, we use a biaffine attention mechanism to capture the span representations while generating the relative position information in the span. Extensive experiments were conducted on five widely-used benchmark datasets to demonstrate the effectiveness of our proposed RoPE-BAM model.