Parsing All: Syntax and Semantics, Dependencies and Spans

Junru Zhou,Zuchao Li,Hai Zhao

DOI: https://doi.org/10.48550/arXiv.1908.11522

2020-10-06

Abstract:Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former. However, few works ever made an attempt to let semantic parsing help syntactic parsing. As linguistic representation formalisms, both syntax and semantics may be represented in either span (constituent/phrase) or dependency, on both of which joint learning was also seldom explored. In this paper, we propose a novel joint model of syntactic and semantic parsing on both span and dependency representations, which incorporates syntactic information effectively in the encoder of neural network and benefits from two representation formalisms in a uniform way. The experiments show that semantics and syntax can benefit each other by optimizing joint objectives. Our single model achieves new state-of-the-art or competitive results on both span and dependency semantic parsing on Propbank benchmarks and both dependency and constituent syntactic parsing on Penn Treebank.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

The problems that this paper attempts to solve are as follows: In existing natural language processing models, syntax and semantics parsing are usually processed independently, and most research focuses on how to use syntax information to assist semantic parsing, while very few works explore how to let semantic parsing in turn assist syntax parsing. Moreover, although both syntax and semantics can be represented by span/constituent or dependency, few studies consider the joint learning of these two representation forms simultaneously.

Specifically, this paper aims to fill the following gaps:
1. **Interaction between syntax and semantics**: Explore whether semantic parsing can in turn assist syntax parsing.
2. **Joint representation forms**: Consider the span/constituent and dependency representation forms of both syntax and semantics simultaneously and perform joint modeling.
3. **Unified framework**: Propose a unified model that can handle syntax and semantics parsing simultaneously to improve the performance of both.

To achieve these goals, the author proposes a new joint model, which effectively combines syntax information in the neural network encoder and utilizes the two representation forms (span/constituent and dependency) in a unified manner. Experimental results show that by optimizing the joint objective function, syntax and semantics parsing can promote each other, thus achieving new state - of - the - art or competitive results on multiple benchmark datasets.

### Formula summary
- **Span/constituent score**:
\[
S(i,j) = W_2g(\text{LN}(W_1s_{ij} + b_1)) + b_2
\]
where \( s_{ij} \) is the representation vector of span/constituent, \( g \) is the ReLU activation function, and \( \text{LN} \) is layer normalization.

- **Dependency score**:
\[
\alpha_{ij} = h_i^T W g_j + U^T h_i + V^T g_j + b
\]
where \( h_i \) and \( g_j \) are vectors calculated by a single - layer perceptron, \( W \), \( U \) and \( V \) are weight matrices and vectors, and \( b \) is a bias term.

- **Semantic role score**:
\[
\Phi_r(p, a) = W_5 g(\text{LN}(W_4 [p_k; a_{ij}] + b_4)) + b_5
\]
where \( p_k \) is the representation of the predicate, \( a_{ij} \) is the representation of the argument, \( g \) is the ReLU activation function, and \( \text{LN} \) is layer normalization.

Through these formulas, the model can simultaneously optimize the objective functions of syntax and semantics parsing during the training process, thereby achieving mutual promotion between the two.