Abstract:Neural network models have recently received heated research attention in the natural language processing community. Compared with traditional models with discrete features, neural models have two main advantages. First, they take low-dimensional, real-valued embedding vectors as inputs, which can be trained over large raw data, thereby addressing the issue of feature sparsity in discrete models. Second, deep neural networks can be used to automatically combine input features, and including non-local features that capture semantic patterns that cannot be expressed using discrete indicator features. As a result, neural network models have achieved competitive accuracies compared with the best discrete models for a range of NLP tasks.On the other hand, manual feature templates have been carefully investigated for most NLP tasks over decades and typically cover the most useful indicator pattern for solving the problems. Such information can be complementary the features automatically induced from neural networks, and therefore combining discrete and neural features can potentially lead to better accuracy compared with models that leverage discrete or neural features only.In this paper, we systematically investigate the effect of discrete and neural feature combination for a range of fundamental NLP tasks based on sequence labeling, including word segmentation, POS tagging and named entity recognition for Chinese and English, respectively. Our results on standard benchmarks show that state-of-the-art neural models can give accuracies comparable to the best discrete models in the literature for most tasks and combing discrete and neural features unanimously yield better results.

A joint method for Chinese word segmentation and part-of-speech labeling based on deep neural network

Joint Segmentation and Tagging with Coupled Sequences Labeling

Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-stage Span Labeling

A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

A Joint Model for Unsupervised Chinese Word Segmentation.

A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

A Joint Segmenting And Labeling Approach For Chinese Lexical Analysis

DAG-based Long Short-Term Memory for Neural Word Segmentation

Long Short-Term Memory Neural Networks for Chinese Word Segmentation.

A Unified Model for Joint Chinese Word Segmentation and Dependency Parsing

A Long Dependency Aware Deep Architecture for Joint Chinese Word Segmentation and POS Tagging.

A Unified Model for Joint Chinese Word Segmentation and POS Tagging with Heterogeneous Annotation Corpora.

Deep Learning for Chinese Word Segmentation and POS Tagging.

Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation

Combining Discrete and Neural Features for Sequence Labeling

State-of-the-art Chinese Word Segmentation with Bi-LSTMs

Neural Networks Incorporating Dictionaries for Chinese Word Segmentation.

Bidirectional LSTM-CRF Attention-based Model for Chinese Word Segmentation

Joint Chinese Word Segmentation and POS Tagging on Heterogeneous Annotated Corpora with Multiple Task Learning.

A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing