Abstract:The recent success of neural networks in NLP applications has provided a strong impetus to develop supervised models for semantic role labeling (SRL) that forego the requirement for extensive feature engineering. Recent state-of-the-art approaches require high-quality annotated datasets that are costly to obtain and almost unavailable for low-resource languages. We present a semi-supervised approach that utilizes both labeled and unlabeled data to provide performance improvement over a mere supervised SRL model. We show that our proposed semi-supervised SRL model provides larger improvement over a supervised model in the scenario where labeled training data size is small. Our SRL system leverages unlabeled data under the language modeling paradigm. We demonstrate that the incorporation of a self pretrained bidirectional language model (S-PrLM) into a SRL system can help in SRL performance improvement by learning composition functions from the unlabeled data. Previous researches have concluded that syntax information is very useful for high-performing SRL systems, so we incorporate syntax information by employing an unsupervised approach to leverage dependency path information to connect argument candidates in vector space, which helps in distinguishing arguments with similar contexts but different syntactic functions. The basic idea is to connect predicate (w(p)) with argument candidate (w(a)) with the dependency path (r) between them in the embedding space. Experiments on the CoNLL-2008 and CoNLL-2009 datasets confirm that our full SRL model outperforms previous best models in terms of F-1 score.

Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies

A Semi-shared Hierarchical Joint Model for Sequence Labeling

Semi-Supervised Semantic Role Labeling with Bidirectional Language Models

Unsupervised Chunking with Hierarchical RNN

Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels

Sequence Labeling with Deep Gated Dual Path CNN.

Semi-supervised Multitask Learning for Sequence Labeling

Empower Sequence Labeling with Task-Aware Neural Language Model

Learning Hierarchical Structures On-The-Fly With A Recurrent-Recursive Model For Sequences

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks.

Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts

Effective Spoken Language Labeling with Deep Recurrent Neural Networks

Semi-Supervised Hierarchical Multi-Label Classifier Based on Local Information

Neural Latent Dependency Model for Sequence Labeling

Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks

A Survey on Recent Advances in Sequence Labeling from Deep Learning Models

Multi-Task Cross-Lingual Sequence Tagging from Scratch

Chinese Semantic Role Labeling with Hierarchical Semantic Knowledge

Dual Learning for Semi-Supervised Natural Language Understanding

Hierarchical Label Inference Incorporating Attribute Semantics in Attributed Networks

Semantic Role Labeling Using Recursive Neural Network