Abstract:In recent years, deep neural networks have achieved significant success in Chinese word segmentation and many other natural language processing tasks. Most of these algorithms are end-to-end trainable systems and can effectively process and learn from large scale labeled datasets. However, these methods typically lack the capability of processing rare words and data whose domains are different from training data. Previous statistical methods have demonstrated that human knowledge can provide valuable information for handling rare cases and domain shifting problems. In this paper, we seek to address the problem of incorporating dictionaries into neural networks for the Chinese word segmentation task. Two different methods that extend the bi-directional long short-term memory neural network are proposed to perform the task. To evaluate the performance of the proposed methods, state-of-the-art supervised models based methods and domain adaptation approaches are compared with our methods on nine datasets from different domains. The experimental results demonstrate that the proposed methods can achieve better performance than other state-of-the-art neural network methods and domain adaptation approaches in most cases.

Long Short-Term Memory Neural Networks for Chinese Word Segmentation.

DAG-based Long Short-Term Memory for Neural Word Segmentation

Neural Word Segmentation Learning for Chinese

Bidirectional LSTM-CRF Attention-based Model for Chinese Word Segmentation

Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation

Hyper-Gated Recurrent Neural Networks for Chinese Word Segmentation.

State-of-the-art Chinese Word Segmentation with Bi-LSTMs

Neural Networks Incorporating Dictionaries for Chinese Word Segmentation.

Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation.

A Deep Convolutional Neural Model for Character-Based Chinese Word Segmentation

Neural Chinese Word Segmentation with Dictionary Knowledge

Chinese Word Segmentation with Maximum Entropy and N-gram Language Model

Unsupervised Neural Word Segmentation for Chinese Via Segmental Language Modeling

Deep Learning for Chinese Word Segmentation and POS Tagging.

Chinese Word Segmentation with Heterogeneous Graph Neural Network

A Long Dependency Aware Deep Architecture for Joint Chinese Word Segmentation and POS Tagging.

Neural Chinese Word Segmentation as Sequence to Sequence Translation

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision

Investigating Self-Attention Network for Chinese Word Segmentation

Unsupervised Word Segmentation with Bi-directional Neural Language Model