Abstract:Dependency syntax has been demonstrated highly useful for a number of natural language processing (NLP) tasks. Typical approaches of utilizing dependency syntax include TreeRNN and Tree-Linearization, both of which exploit explicit 1-best tree outputs from a welltrained parser as inputs. However, these approaches may suffer from error propagation due to the inevitable errors contained in the 1-best tree outputs. In this work, we propose a novel approach to integrate dependency syntax without using the discrete tree outputs. The key idea is to use the intermediate hidden representations of a welltrained encoder-decoder dependency parser, which are referred to as Dependency-based Syntax-Aware Word Representations (Dep-SAWRs). Then, we simply concatenate such DepSAWRs with the conventional context-insensitive word embeddings to compose input word representations, without requiring to modify the model architecture of the downstream tasks. We evaluate the proposed method on four kinds of typical NLP tasks, including sentence classification, sentence matching, sequence labeling and machine translation. Experimental results show that the proposed approach is highly promising. On the one hand, it can utilize dependency syntax effectively, bringing consistently better performance on the four tasks compared with baselines without using syntax. On the other hand, the proposed method can outperform the Tree-RNN and Tree-Linearization approaches in most settings, and meanwhile are highly efficient in syntax integration. In addition, the proposed method would be easily extendable to encoding other structural attributes of language. (C) 2020 Elsevier B.V. All rights reserved.

Dependency Parsing with Short Dependency Relations in Unlabeled Data

Using Short Dependency Relations from Auto-Parsed Data for Chinese Dependency Parsing

Chinese Dependency Parsing Based on Treebank

Research on Data-driven Dependency Parsing

Textless Dependency Parsing by Labeled Sequence Prediction

Building Powerful Dependency Parsers for Resource-Poor Languages

A Practical Chinese Dependency Parser Based on A Large-scale Dataset

Universal Dependencies Parsing For Colloquial Singaporean English

A Two-Stage Parser for Multilingual Dependency Parsing

Probabilistic Model for Syntactic and Semantic Dependency Parsing

Improved Dependency Parsing Using Implicit Word Connections Learned from Unlabeled Data.

A Statistical Dependency Parser of Chinese under Small Training Data *

A Fast and Lightweight System for Multilingual Dependency Parsing.

Semantic role labeling for dependency relations

Dependency Parsing with Noisy Multi-annotation Data

Multi-level Chunk-based Constituent-to-Dependency Treebank Transformation for Tibetan Dependency Parsing

Improving Distant Supervised Relation Extraction Using Shortest Dependency Path

A Survey of Unsupervised Dependency Parsing.

Dependency Tree-based SRL with Proper Pruning and Extensive Feature Engineering.

Cross Language Dependency Parsing Using a Bilingual Lexicon.

Dependency-based syntax-aware word representations