Abstract:Words provide a useful source of information for Chinese NLP, and word segmentation has been taken as a pre-processing step for most downstream tasks. For many NLP tasks, however, word segmentation can introduce noise and lead to error propagation. The rise of neural representation learning models allows sentence-level semantic information to be collected from characters directly. As a result, it is an empirical question whether a fully character-based model should be used instead of first performing word segmentation. We investigate a neural representation that simultaneously encodes character and word information without the need for segmentation. In particular, candidate words are found in a sentence by matching with a pre-defined lexicon. A lattice structured LSTM is used to encode the resulting word-character lattice, where gate vectors are used to control information flow through words, so that the more useful words can be automatically identified by end-to-end training. We compare the performance of the resulting lattice LSTM and baseline sequence LSTM structures over both character sequences and automatically segmented word sequences. Results on NER show that the character-word lattice model can significantly improve the performance. In addition, as a general sentence representation architecture, character-word lattice LSTM can also be used for learning contextualized representations. To this end, we compare lattice LSTM structure with its sequential LSTM counterpart, namely ELMo. Results show that our lattice version of ELMo gives better language modeling performances. On Chinese POS-tagging, chunking and syntactic parsing tasks, the resulting contextualized Chinese embeddings also give better performance than ELMo trained on the same data.

Leveraging morphological information via employing word hashing for sequence labeling

Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling

Empower Sequence Labeling with Task-Aware Neural Language Model

Lattice LSTM for Chinese Sentence Representation

Chinese word segmentation as morpheme-based lexical chunking

A Hybrid Model for Computational Morphology Application

A morphology-based Chinese word segmentation method

Combining Discrete and Neural Features for Sequence Labeling

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

A Joint Segmenting And Labeling Approach For Chinese Lexical Analysis

A Local Information Perception Enhancement–Based Method for Chinese NER

Learning Task-Specific Representation for Novel Words in Sequence Labeling.

Parsing-based Chinese word segmentation integrating morphological and syntactic information

Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models

Learning Effective Word Embedding Using Morphological Word Similarity

Improving Word Representations with Document Labels

Evaluating the Utility of Hand-crafted Features in Sequence Labelling

Co-learning of Word Representations and Morpheme Representations.

Hybrid Semi-Markov CRF for Neural Sequence Labeling.

Semantic Role Labeling Integrated with Multilevel Linguistic Cues and Bi-LSTM-CRF

A joint method for Chinese word segmentation and part-of-speech labeling based on deep neural network