Abstract:Chinese word segmentation is an active area in Chinese language processing though it is suffering from the argument about what precisely is a word in Chinese. Based on corpus-based segmentation standard, we launched this study. In detail, we regard Chinese word segmentation as a character-based tagging problem. We show that there has been a potent trend of using a character-based tagging approach in this field. In particular, learning from segmented corpus with or without additional linguistic resources is treated in a unified way in which the only difference depends on how the feature template set is selected. It differs from existing work in that both feature template selection and tag set selection are considered in our approach, instead of the previous feature template focus only technique. We show that there is a significant performance difference as different tag sets are selected. This is especially applied to a six-tag set, which is good enough for most current segmented corpora. The linguistic meaning of a tag set is also discussed. Our results show that a simple learning system with six n -gram feature templates and a six-tag set can obtain competitive performance in the cases of learning only from a training corpus. In cases when additional linguistic resources are available, an ensemble learning technique, assistant segmenter, is proposed and its effectiveness is verified. Assistant segmenter is also proven to be an effective method as segmentation standard adaptation that outperforms existing ones. Based on the proposed approach, our system provides state-of-the-art performance in all 12 corpora of three international Chinese word segmentation bakeoffs.

A Comparison Study of Candidate Generation for Chinese Word Segmentation

Unified Framework of Performing Chinese Word Segmentation and Part-Of-Speech Tagging

A Comparative Study on Chinese Word Segmentation Using Statistical Models

A Joint Model for Unsupervised Chinese Word Segmentation.

A Comparison Study of Sequence Labeling Methods for Chinese Word Segmentation,POS Tagging Models

Multi-Grained Chinese Word Segmentation.

A Local Generative Model For Chinese Word Segmentation

A Pragmatic Approach for Classical Chinese Word Segmentation.

Word-based and Character-Based Word Segmentation Models: Comparison and Combination

A realistic and robust model for Chinese word segmentation

A New Unsupervised Approach to Word Segmentation.

Exploiting Unlabeled Text with Different Unsupervised Segmentation Criteria for Chinese Word Segmentation

A New Error-driven Learning Approach for Chinese Word Segmentation

Chinese Word Segmentation based on Word boundary Classificatioin

A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation

A Unified Character-Based Tagging Framework for Chinese Word Segmentation.

Switch-LSTMs for Multi-Criteria Chinese Word Segmentation.

How to Enhance Chinese Word Segmentation Using Knowledge Graphs

RethinkCWS: is Chinese Word Segmentation a Solved Task?

A Simple and Effective Unsupervised Word Segmentation Approach.

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision