Abstract:Two important sequence tasks are sequence modeling and labeling. Sequence modeling involves determining the probabilities of sequences, e.g. language modeling. It is still difficult to improve language modeling with additional relevant tags, e.g. part-of-speech (POS) tags. For sequence labeling, it is worthwhile to explore task-dependent semi-supervised learning to leverage a mix of labeled and unlabeled data, besides pre-training. In this paper, we propose to upgrade condtional random fields (CRFs) and obtain a joint generative model of observation and label sequences, called joint random fields (JRFs). Specifically, we propose to use the potential function in the original CRF as the potential function that defines the joint distribution. This development from CRFs to JRFs benefits both modeling and labeling of sequence data, as shown in our experiments. For example, the JRF model (using POS tags) outperforms traditional language models and avoids the need to produce hypothesized labels by a standalone POS tagger. For sequence labeling, task-dependent semi-supervised learning by JRFs consistently outperform the CRF baseline and self-training, on POS tagging, chunking and NER.

Conditional Random Fields Based Label Sequence and Information Feedback

Conditional Random Fields Based POS Tagging

Improving Sequence Tagging Using Machine-Learning Techniques

The Application of CRFs in Part-of-Speech Tagging

Chinese Semantic Role Labeling Based on Conditional Random Fields

Labeling Sequential Data Based on Word Representations and Conditional Random Fields

Upgrading CRFS to JRFS and Its Benefits to Sequence Modeling and Labeling.

Applying Conditional Random Fields to Chinese Shallow Parsing

A Chinese Part-of-speech Tagging Approach Using Conditional Random Fields

Masked Conditional Random Fields for Sequence Labeling

Conditional random fields and its application to language analysis system

Semi-Markov Conditional Random Fields for sequence labeling

AN UNSUPERVISED CHINESE PART-OF-SPEECH TAGGING APPROACH USING CONDITIONAL RANDOM FIELDS

Chinese Text Chunking Based CRF

A CRF Sequence Labeling Approach to Chinese Punctuation Prediction.

Chinese Named Entity Recognition with the Improved Smoothed Conditional Random Fields

Applying conditional random fields on Chinese syllable recognition

Hybrid Semi-Markov CRF for Neural Sequence Labeling.

Sparse Higher Order Conditional Random Fields for Improved Sequence Labeling.

Mongolian Part-of-speech Tagging Approach Based on Conditional Random Fields

Automatic Labeling of Semantic Role on Chinese FrameNet Using Conditional Random Fields.