Abstract:Recent state-of-the-art (SOTA) effective neural network methods and fine-tuning methods based on pre-trained models (PTM) have been used in Chinese word segmentation (CWS), and they achieve great results. However, previous works focus on training the models with the fixed corpus at every iteration. The intermediate generated information is also valuable. Besides, the robustness of the previous neural methods is limited by the large-scale annotated data. There are a few noises in the annotated corpus. Limited efforts have been made by previous studies to deal with such problems. In this work, we propose a self-supervised CWS approach with a straightforward and effective architecture. First, we train a word segmentation model and use it to generate the segmentation results. Then, we use a revised masked language model (MLM) to evaluate the quality of the segmentation results based on the predictions of the MLM. Finally, we leverage the evaluations to aid the training of the segmenter by improved minimum risk training. Experimental results show that our approach outperforms previous methods on 9 different CWS datasets with single criterion training and multiple criteria training and achieves better robustness.(1)

A Deep Convolutional Neural Model for Character-Based Chinese Word Segmentation

Multiple Character Embeddings for Chinese Word Segmentation

Closed-Set Chinese Word Segmentation Based On Convolutional Neural Network Model

Deep Learning for Chinese Word Segmentation and POS Tagging.

Neural Chinese Word Segmentation as Sequence to Sequence Translation

Neural Word Segmentation Learning for Chinese

A Gap-Based Framework for Chinese Word Segmentation Via Very Deep Convolutional Networks

Long Short-Term Memory Neural Networks for Chinese Word Segmentation.

Bidirectional LSTM-CRF Attention-based Model for Chinese Word Segmentation

Word-Context Character Embeddings for Chinese Word Segmentation.

A Joint Model for Unsupervised Chinese Word Segmentation.

Neural Chinese Word Segmentation with Dictionary Knowledge

Effective Neural Solution for Multi-criteria Word Segmentation

Feature-based Neural Language Model and Chinese Word Segmentation.

Unsupervised Neural Word Segmentation for Chinese Via Segmental Language Modeling

Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

Chinese Word Segmentation with Heterogeneous Graph Neural Network

Robust Chinese Word Segmentation with Contextualized Word Representations

DAG-based Long Short-Term Memory for Neural Word Segmentation

Improving Cross-Domain Chinese Word Segmentation with Word Embeddings

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision