Abstract:Chinese word segmentation is an active area in Chinese language processing though it is suffering from the argument about what precisely is a word in Chinese. Based on corpus-based segmentation standard, we launched this study. In detail, we regard Chinese word segmentation as a character-based tagging problem. We show that there has been a potent trend of using a character-based tagging approach in this field. In particular, learning from segmented corpus with or without additional linguistic resources is treated in a unified way in which the only difference depends on how the feature template set is selected. It differs from existing work in that both feature template selection and tag set selection are considered in our approach, instead of the previous feature template focus only technique. We show that there is a significant performance difference as different tag sets are selected. This is especially applied to a six-tag set, which is good enough for most current segmented corpora. The linguistic meaning of a tag set is also discussed. Our results show that a simple learning system with six n -gram feature templates and a six-tag set can obtain competitive performance in the cases of learning only from a training corpus. In cases when additional linguistic resources are available, an ensemble learning technique, assistant segmenter, is proposed and its effectiveness is verified. Assistant segmenter is also proven to be an effective method as segmentation standard adaptation that outperforms existing ones. Based on the proposed approach, our system provides state-of-the-art performance in all 12 corpora of three international Chinese word segmentation bakeoffs.

How unsupervised learning affects character tagging based Chinese Word Segmentation: A quantitative investigation

CRF with Locality-Consistent Dictionary Learning for Semantic Segmentation

Incorporating Global Information into Supervised Learning for Chinese Word Segmentation

Unsupervised Segmentation Helps Supervised Learning of Character Tagging for Word Segmentation and Named Entity Recognition.

Unsupervised Learning helps Supervised Neural Word Segmentation

Exploiting Unlabeled Text with Different Unsupervised Segmentation Criteria for Chinese Word Segmentation

Improving Chinese Word Segmentation with Description Length Gain

Enhancing Chinese Word Segmentation Using Unlabeled Data

Integrating Unsupervised and Supervised Word Segmentation: the Role of Goodness Measures

Unsupervised segmentation of chinese corpus using accessor variety

A Unified Character-Based Tagging Framework for Chinese Word Segmentation.

Enhancing Chinese Word Segmentation With Character Clustering

Lexicon-based Semi-Crf for Chinese Clinical Text Word Segmentation

Effective Tag Set Selection In Chinese Word Segmentation Via Conditional Random Field Modeling

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision

Neural Word Segmentation Learning for Chinese

CRFs Based Chinese Word Segmentation

Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation

A Local Generative Model For Chinese Word Segmentation

LM Enhanced BiRNN-CRF for Joint Chinese Word Segmentation and POS Tagging.

Chinese Word Segmentation Via BiLSTM+Semi-CRF with Relay Node