Abstract:Neural network models have recently received heated research attention in the natural language processing community. Compared with traditional models with discrete features, neural models have two main advantages. First, they take low-dimensional, real-valued embedding vectors as inputs, which can be trained over large raw data, thereby addressing the issue of feature sparsity in discrete models. Second, deep neural networks can be used to automatically combine input features, and including non-local features that capture semantic patterns that cannot be expressed using discrete indicator features. As a result, neural network models have achieved competitive accuracies compared with the best discrete models for a range of NLP tasks.On the other hand, manual feature templates have been carefully investigated for most NLP tasks over decades and typically cover the most useful indicator pattern for solving the problems. Such information can be complementary the features automatically induced from neural networks, and therefore combining discrete and neural features can potentially lead to better accuracy compared with models that leverage discrete or neural features only.In this paper, we systematically investigate the effect of discrete and neural feature combination for a range of fundamental NLP tasks based on sequence labeling, including word segmentation, POS tagging and named entity recognition for Chinese and English, respectively. Our results on standard benchmarks show that state-of-the-art neural models can give accuracies comparable to the best discrete models in the literature for most tasks and combing discrete and neural features unanimously yield better results.

Design Challenges and Misconceptions in Neural Sequence Labeling.

Visualizing and Understanding Neural Models in NLP

Citation Metadata Extraction Via Deep Neural Network-based Segment Sequence Labeling

A Survey on Recent Advances in Sequence Labeling from Deep Learning Models

Neural Models for Sequence Chunking

Combining Discrete and Neural Features for Sequence Labeling

Empower Sequence Labeling with Task-Aware Neural Language Model

Sequence Labeling with Meta-Learning

Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Tackling Sequence to Sequence Mapping Problems with Neural Networks

Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks.

A Comparison and Improvement of Online Learning Algorithms for Sequence Labeling.

Evaluating Neural Morphological Taggers for Sanskrit

A Survey of Label-noise Representation Learning: Past, Present and Future

Label Attention Network for Structured Prediction

Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise.

Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models

Chinese Syllable-to-character Conversion with Recurrent Neural Network Based Supervised Sequence Labelling

Adaptive Self-training for Few-shot Neural Sequence Labeling