Abstract:Words provide a useful source of information for Chinese NLP, and word segmentation has been taken as a pre-processing step for most downstream tasks. For many NLP tasks, however, word segmentation can introduce noise and lead to error propagation. The rise of neural representation learning models allows sentence-level semantic information to be collected from characters directly. As a result, it is an empirical question whether a fully character-based model should be used instead of first performing word segmentation. We investigate a neural representation that simultaneously encodes character and word information without the need for segmentation. In particular, candidate words are found in a sentence by matching with a pre-defined lexicon. A lattice structured LSTM is used to encode the resulting word-character lattice, where gate vectors are used to control information flow through words, so that the more useful words can be automatically identified by end-to-end training. We compare the performance of the resulting lattice LSTM and baseline sequence LSTM structures over both character sequences and automatically segmented word sequences. Results on NER show that the character-word lattice model can significantly improve the performance. In addition, as a general sentence representation architecture, character-word lattice LSTM can also be used for learning contextualized representations. To this end, we compare lattice LSTM structure with its sequential LSTM counterpart, namely ELMo. Results show that our lattice version of ELMo gives better language modeling performances. On Chinese POS-tagging, chunking and syntactic parsing tasks, the resulting contextualized Chinese embeddings also give better performance than ELMo trained on the same data.

Generation of syllable-lattice based on active state in Mandarin continuous speech recognition

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

A Trellis Based Fast Lattice Generating Algorithm

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Speech Recognition Lattice-Generating Algorithm with Forward-Backward Language Model

Lattice LSTM for Chinese Sentence Representation

Automatic segmentation of Chinese Mandarin speech into syllable-like

A Method for Syllable Segmentation in Mandarin Speech Recognition

Improving the Syllable-Synchronous Network Search Algorithm for Word Decoding in Continuous Chinese Speech Recognition

Context Dependent Syllable Acoustic Model For Continuous Chinese Speech Recognition

Lattice-based Indexing for Spontaneous Mandarin Speech

Lattice-based continuous Chinese speech indexing based on mutual information confidence measure

A Study of Lattice-Based Spoken Term Detection for Chinese Spontaneous Speech

Subword Encoding in Lattice LSTM for Chinese Word Segmentation

A method for Chinese syllables recognition based upon sub-syllable hidden Markov model

Syllable HMM Based Mandarin TTS and Comparison with Concatenative TTS.

Feature Selection In Mandarin Large Vocabulary Continuous Speech Recognition

Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model

Visual Information Assisted Mandarin Large Vocabulary Continuous Speech Recognition

Phone Lattice Reconstruction for Embedded Language Recognition in LVCSR

Fusing Multiple Systems into a Compact Lattice Index for Chinese Spoken Term Detection.