Abstract:Neural network models have recently received heated research attention in the natural language processing community. Compared with traditional models with discrete features, neural models have two main advantages. First, they take low-dimensional, real-valued embedding vectors as inputs, which can be trained over large raw data, thereby addressing the issue of feature sparsity in discrete models. Second, deep neural networks can be used to automatically combine input features, and including non-local features that capture semantic patterns that cannot be expressed using discrete indicator features. As a result, neural network models have achieved competitive accuracies compared with the best discrete models for a range of NLP tasks.On the other hand, manual feature templates have been carefully investigated for most NLP tasks over decades and typically cover the most useful indicator pattern for solving the problems. Such information can be complementary the features automatically induced from neural networks, and therefore combining discrete and neural features can potentially lead to better accuracy compared with models that leverage discrete or neural features only.In this paper, we systematically investigate the effect of discrete and neural feature combination for a range of fundamental NLP tasks based on sequence labeling, including word segmentation, POS tagging and named entity recognition for Chinese and English, respectively. Our results on standard benchmarks show that state-of-the-art neural models can give accuracies comparable to the best discrete models in the literature for most tasks and combing discrete and neural features unanimously yield better results.

Neural Parse Combination

K-Best Combination of Syntactic Parsers.

A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing.

Neural Language Modeling by Jointly Learning Syntax and Lexicon.

Parsing Penn Chinese Treebank Based on Lexicalized Model

Parsing All: Syntax and Semantics, Dependencies and Spans

An Effective Neural Network Model for Graph-based Dependency Parsing.

Combining Discrete and Neural Features for Sequence Labeling

Neural System Combination For Machine Translation

Neural correlates of semantic-driven syntactic parsing in sentence comprehension

A Neural Transition-Based Approach for Semantic Dependency Graph Parsing

Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach

Two-stage approach to full Chinese parsing

Joint POS Tagging and Dependence Parsing With Transition-Based Neural Networks

Joint POS Tagging and Dependency Parsing with Transition-based Neural Networks

Improving Feature-Rich Transition-Based Constituent Parsing Using Recurrent Neural Networks.

Improved Discourse Parsing with Two-Step Neural Transition-Based Model

Research on Strategy of Integrating Chinese Lexical Analysis and Parser

Neural semantic parsing

Paraphrase Recognition Via Combination of Neural Classifier and Keywords

Two Local Models for Neural Constituent Parsing