Abstract:Conditional random fields (CRFs) have been widely used for sequence labeling tasks in the field of natural language processing. However, how to model both local and global dependencies among labels is not well solved yet. In this study, we introduce a novel two-stage label decoding method to better model the short- and long-term label dependencies, while being much more computationally efficient with the use of graphics processing units (GPUs). A base model is first used to propose draft labels, and then a novel two-stream self-attention model makes refinements on these draft predictions based on long-range label dependencies. Besides, in order to mitigate the side effects of incorrect draft labels, Bayesian neural networks are used to indicate the labels with high probabilities of being wrong, which helps to mitigate the error propagation. Not only can our method model sentence-level label dependencies, but it is also easily extended to document-level sequence labeling by querying and storing a key-value memory matrix with label co-occurrence relationships. The experimental results on both sentence-level and document-level sequence labeling benchmarks show that the proposed method outperforms existing label decoding methods while taking advantage of parallel computations on GPUs.

Conditional random fields and its application to language analysis system

Conditional Random Fields Based Label Sequence and Information Feedback

Labeling Sequential Data Based on Word Representations and Conditional Random Fields

Chinese Semantic Role Labeling Based on Conditional Random Fields

Chinese Chunking Algorithm Based On Conditional Random Fields

Chinese chunking method based on conditional random fields and semantic classes

Conditional Random Fields for Image Labeling

Gradual Transition Detection with Conditional Random Fields.

Analyzing Sequence Data Based on Conditional Random Fields with Co-Training

Learning Conditional Random Fields From Unaligned Data For Natural Language Understanding

Masked Conditional Random Fields for Sequence Labeling

A conditional random field model for image parsing.

The Application of CRFs in Part-of-Speech Tagging

Automatic Labeling of Semantic Role on Chinese FrameNet Using Conditional Random Fields.

Large-scale Conditional Random Field for Natural Outdoor Scene Labeling

Uncertainty-Aware Label Refinement for Sequence Labeling

Conditional Random Fields Based POS Tagging

Conditional Random Fields for Image Region Labeling with Global Observation.

Uncertainty-Aware Sequence Labeling

A Chinese Part-of-speech Tagging Approach Using Conditional Random Fields

Sparse Higher Order Conditional Random Fields for Improved Sequence Labeling.