Abstract:<p>Natural language processing (NLP) is useful for handling text and speech, and sequence labeling plays an important role by automatically analyzing a sequence (text) to assign category labels to each part. However, the performance of these conventional models depends greatly on hand-crafted features and task-specific knowledge, which is a time consuming task. Several conditional random fields (CRF)-based models for sequence labeling have been presented, but the major limitation is how to use neural networks for extracting useful representations for each unit or segment in the input sequence. In this paper, we propose an attention segmental recurrent neural network (ASRNN) that relies on a hierarchical attention neural semi-Markov conditional random fields (semi-CRF) model for the task of sequence labeling. Our model uses a hierarchical structure to incorporate character-level and word-level information and applies an attention mechanism to both levels. This enables our method to differentiate more important information from less important information when constructing the segmental representation. We evaluated our model on three sequence labeling tasks, including named entity recognition (NER), chunking, and reference parsing. Experimental results show that the proposed model benefited from the hierarchical structure, and it achieved competitive and robust performance on all three sequence labeling tasks.</p>

Attention-Based Recurrent Neural Network for Sequence Labeling.

ASRNN: A recurrent neural network with an attention model for sequence labeling

Label Attention Network for Structured Prediction

Citation Metadata Extraction Via Deep Neural Network-based Segment Sequence Labeling

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks.

Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Sequence Labeling with Deep Gated Dual Path CNN.

Order-Free RNN With Visual Attention for Multi-Label Classification

Label-Dependencies Aware Recurrent Neural Networks

Recurrent Attention Unit

Scene Parsing via Dense Recurrent Neural Networks with Attentional Selection

Deep Neural Network with Attention Model for Scene Text Recognition.

RNN-Based Sequence-Preserved Attention for Dependency Parsing.

Label Attention Network for sequential multi-label classification: you were looking at a wrong self-attention

A Label-Specific Attention-Based Network with Regularized Loss for Multi-label Classification

End-to-End Instance Segmentation with Recurrent Attention

A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding.

Attention-Based Hierarchical Recurrent Neural Network for Phenotype Classification.

DAG-Recurrent Neural Networks for Scene Labeling.

Recurrent Attention Networks for Long-text Modeling