Abstract:In recent years, parsing performance is dramatically improved on in-domain texts thanks to the rapid progress of deep neural network models. The major challenge for current parsing research is to improve parsing performance on out-of-domain texts that are very different from the in-domain training data when there is only a small-scale out-domain labeled data. To deal with this problem, we propose to improve the contextualized word representations via adversarial learning and fine-tuning BERT processes. Concretely, we apply adversarial learning to three representative semi-supervised domain adaption methods, i.e., direct concatenation (CON), feature augmentation (FA), and domain embedding (DE) with two useful strategies, i.e., fused target-domain word representations and orthogonality constraints, thus enabling to model more pure yet effective domain-specific and domain-invariant representations. Simultaneously, we utilize a large-scale target-domain unlabeled data to fine-tune BERT with only the language model loss, thus obtaining reliable contextualized word representations that benefit for the cross-domain dependency parsing. Experiments on a benchmark dataset show that our proposed adversarial approaches achieve consistent improvement, and fine-tuning BERT further boosts parsing accuracy by a large margin. Our single model achieves the same state-of-the-art performance as the top submitted system in the NLPCC-2019 shared task, which uses ensemble models and BERT.

Domain Information Enhanced Dependency Parser.

Cross-Domain Transfer Learning for Dependency Parsing.

Overview of the NLPCC 2019 Shared Task: Cross-Domain Dependency Parsing

Cross-lingual Dependency Parsing As Domain Adaptation.

Learning Domain Differences Automatically for Dependency Parsing Adaptation.

APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing

Semi-supervised Domain Adaptation for Dependency Parsing

Unsupervised Domain Adaptation Method with Semantic-Structural Alignment for Dependency Parsing

Semi-supervised Domain Adaptation for Dependency Parsing Via Improved Contextualized Word Representations.

A Practical Chinese Dependency Parser Based on A Large-scale Dataset

Self-attentive Biaffine Dependency Parsing.

Domain Adaptation for Semantic Parsing.

Semi-supervised Domain Adaptation for Dependency Parsing with Dynamic Matching Network

Adversarial Domain Adaptation For Chinese Semantic Dependency Graph Parsing

Tri-training for Dependency Parsing Domain Adaptation

Research on Data-driven Dependency Parsing

Learning Domain Invariant Word Representations for Parsing Domain Adaptation

Dependency Parsing with Noisy Multi-annotation Data

Chinese Dependency Parsing Based on Treebank

Neural Network based Deep Transfer Learning for Cross-domain Dependency Parsing

Hit dependency parsing: Bootstrap aggregating heterogeneous parsers