Abstract:Words provide a useful source of information for Chinese NLP, and word segmentation has been taken as a pre-processing step for most downstream tasks. For many NLP tasks, however, word segmentation can introduce noise and lead to error propagation. The rise of neural representation learning models allows sentence-level semantic information to be collected from characters directly. As a result, it is an empirical question whether a fully character-based model should be used instead of first performing word segmentation. We investigate a neural representation that simultaneously encodes character and word information without the need for segmentation. In particular, candidate words are found in a sentence by matching with a pre-defined lexicon. A lattice structured LSTM is used to encode the resulting word-character lattice, where gate vectors are used to control information flow through words, so that the more useful words can be automatically identified by end-to-end training. We compare the performance of the resulting lattice LSTM and baseline sequence LSTM structures over both character sequences and automatically segmented word sequences. Results on NER show that the character-word lattice model can significantly improve the performance. In addition, as a general sentence representation architecture, character-word lattice LSTM can also be used for learning contextualized representations. To this end, we compare lattice LSTM structure with its sequential LSTM counterpart, namely ELMo. Results show that our lattice version of ELMo gives better language modeling performances. On Chinese POS-tagging, chunking and syntactic parsing tasks, the resulting contextualized Chinese embeddings also give better performance than ELMo trained on the same data.

Closed-Set Chinese Word Segmentation Based On Convolutional Neural Network Model

A Deep Convolutional Neural Model for Character-Based Chinese Word Segmentation

A Gap-Based Framework for Chinese Word Segmentation Via Very Deep Convolutional Networks

Neural Word Segmentation Learning for Chinese

Long Short-Term Memory Neural Networks for Chinese Word Segmentation.

Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation.

Neural Chinese Word Segmentation with Dictionary Knowledge

Hyper-Gated Recurrent Neural Networks for Chinese Word Segmentation.

Chinese Word Segmentation with Maximum Entropy and N-gram Language Model

Feature-based Neural Language Model and Chinese Word Segmentation.

Deep Learning for Chinese Word Segmentation and POS Tagging.

Fast Neural Chinese Word Segmentation for Long Sentences

A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

Unsupervised Neural Word Segmentation for Chinese Via Segmental Language Modeling

Fast and Accurate Neural Word Segmentation for Chinese

Lattice LSTM for Chinese Sentence Representation

Neural Chinese Word Segmentation as Sequence to Sequence Translation

Word-Context Character Embeddings for Chinese Word Segmentation.

Robust Chinese Word Segmentation with Contextualized Word Representations

Max-Margin Tensor Neural Network for Chinese Word Segmentation

LM Enhanced BiRNN-CRF for Joint Chinese Word Segmentation and POS Tagging.