Position-aware Self-Attention Based Neural Sequence Labeling

Wei,Zanbo Wang,Xianling Mao,Guangyou Zhou,Pan Zhou,Sheng Jiang
DOI: https://doi.org/10.1016/j.patcog.2020.107636
IF: 8
2020-01-01
Pattern Recognition
Abstract:•This paper identifies the problem of modeling discrete context dependencies in sequence labeling tasks.•This paper develops a well-designed self-attentional context fusion network to provide complementary context information on the basis of Bi-LSTM.•This paper proposes a novel position-aware self-attention to incorporate three different positional factors for exploring the relative position information among token.•The proposed model achieves state-of-the-arts performance on part-of-speech (POS) tagging, named entity recognition (NER) and phrase chunking tasks.
What problem does this paper attempt to address?