Abstract:The limited scale and genre coverage of labeled data greatly hinders the effectiveness of supervised models, especially when analyzing spoken languages, such as texts transcribed from speech and informal text including tweets and product comments in Internet. In order to effectively utilize multiple labeled datasets with heterogeneous annotations for the same task, this paper proposes a coupled sequence labeling model that can directly learn and infer two heterogeneous annotations simultaneously, using Chinese part-of-speech (POS) tagging as our case study. The key idea is to bundle two sets of POS tags together (e.g., “$[{NN},{n}$ ]”), and build a conditional random field (CRF) based tagging model in the enlarged space of bundled tags with the help of ambiguous labeling. To train our model on two nonoverlapping datasets that each has only one-side tags, we transform a one-side tag into a set of bundled tags by concatenating the tag with every possible tag at the missing side according to a predefined context-free tag-to-tag mapping function, thus producing ambiguous labeling as weak supervision. We design and investigate four different context-free tag-to-tag mapping functions, and find out that the coupled model achieves its best performance when each one-side tag is mapped to all tags at the other side (namely complete mapping), indicating that the model can effectively learn the loose mapping between the two heterogeneous annotations, without the need of manually designed mapping rules. Moreover, we propose a context-aware online pruning strategy that can more accurately capture mapping relationships between annotations based on contextual evidences and thus effectively solve the severe inefficiency problem with our coupled model under complete mapping, making it comparable with the baseline CRF model. Experiments on benchmark datasets show that our coupled model significantly outperforms the state-of-the-art baselines on both one-side POS tagging and annotation conversion tasks. The codes and newly annotated data are released for research usage.11 http://hlt.suda.edu.cn/∼zhli.

POS Tag-enhanced Coarse-to-fine Attention for Neural Machine Translation

Neural Machine Translation with Target-Attention Model.

Enhanced Neural Machine Translation by Joint Decoding with Word and POS-tagging Sequences.

Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing?

Optimizing Attention Mechanism for Neural Machine Transltion

Syntax-Directed Attention for Neural Machine Translation

Learning When to Attend for Neural Machine Translation

Joint Optimization for Chinese POS Tagging and Dependency Parsing

Interactive Attention for Neural Machine Translation

Neural Machine Translation with Supervised Attention

Fine-grained attention mechanism for neural machine translation

Predicting Target Language CCG Supertags Improves Neural Machine Translation

Joint Models for Chinese POS Tagging and Dependency Parsing.

Enhancing low-resource neural machine translation with syntax-graph guided self-attention

Coupled POS Tagging on Heterogeneous Annotations

Effective Approaches to Attention-based Neural Machine Translation

A Separately Passive-Aggressive Training Algorithm for Joint POS Tagging and Dependency Parsing

Temporal Attention Model for Neural Machine Translation

Modeling Coverage for Neural Machine Translation

Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging

Chinese POS Tagging Based on Maximum Entropy Model