Abstract:Chinese word segmentation is an active area in Chinese language processing though it is suffering from the argument about what precisely is a word in Chinese. Based on corpus-based segmentation standard, we launched this study. In detail, we regard Chinese word segmentation as a character-based tagging problem. We show that there has been a potent trend of using a character-based tagging approach in this field. In particular, learning from segmented corpus with or without additional linguistic resources is treated in a unified way in which the only difference depends on how the feature template set is selected. It differs from existing work in that both feature template selection and tag set selection are considered in our approach, instead of the previous feature template focus only technique. We show that there is a significant performance difference as different tag sets are selected. This is especially applied to a six-tag set, which is good enough for most current segmented corpora. The linguistic meaning of a tag set is also discussed. Our results show that a simple learning system with six n -gram feature templates and a six-tag set can obtain competitive performance in the cases of learning only from a training corpus. In cases when additional linguistic resources are available, an ensemble learning technique, assistant segmenter, is proposed and its effectiveness is verified. Assistant segmenter is also proven to be an effective method as segmentation standard adaptation that outperforms existing ones. Based on the proposed approach, our system provides state-of-the-art performance in all 12 corpora of three international Chinese word segmentation bakeoffs.

Unified Multi-Criteria Chinese Word Segmentation with BERT

A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder.

Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation

Effective Neural Solution for Multi-criteria Word Segmentation

A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation

Adversarial Multi-Criteria Learning for Chinese Word Segmentation

BERT Meets Chinese Word Segmentation

Switch-LSTMs for Multi-Criteria Chinese Word Segmentation.

Chinese Word Segmentation based on Word boundary Classificatioin

Improving Multi-model Hybrid Chinese Long-text Classification Through BERT Optimisation.

A Joint Model for Unsupervised Chinese Word Segmentation.

Pre-training with Meta Learning for Chinese Word Segmentation.

A Unified Character-Based Tagging Framework for Chinese Word Segmentation.

Bidirectional LSTM-CRF Attention-based Model for Chinese Word Segmentation

MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining

LM Enhanced BiRNN-CRF for Joint Chinese Word Segmentation and POS Tagging.

Word Segmentation on Micro-blog Texts with External Lexicon and Heterogeneous Data.

A Deep Convolutional Neural Model for Character-Based Chinese Word Segmentation

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision

Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models