Abstract:Words provide a useful source of information for Chinese NLP, and word segmentation has been taken as a pre-processing step for most downstream tasks. For many NLP tasks, however, word segmentation can introduce noise and lead to error propagation. The rise of neural representation learning models allows sentence-level semantic information to be collected from characters directly. As a result, it is an empirical question whether a fully character-based model should be used instead of first performing word segmentation. We investigate a neural representation that simultaneously encodes character and word information without the need for segmentation. In particular, candidate words are found in a sentence by matching with a pre-defined lexicon. A lattice structured LSTM is used to encode the resulting word-character lattice, where gate vectors are used to control information flow through words, so that the more useful words can be automatically identified by end-to-end training. We compare the performance of the resulting lattice LSTM and baseline sequence LSTM structures over both character sequences and automatically segmented word sequences. Results on NER show that the character-word lattice model can significantly improve the performance. In addition, as a general sentence representation architecture, character-word lattice LSTM can also be used for learning contextualized representations. To this end, we compare lattice LSTM structure with its sequential LSTM counterpart, namely ELMo. Results show that our lattice version of ELMo gives better language modeling performances. On Chinese POS-tagging, chunking and syntactic parsing tasks, the resulting contextualized Chinese embeddings also give better performance than ELMo trained on the same data.

Learning Spoken Language Representations with Neural Lattice Language Modeling

Lattice LSTM for Chinese Sentence Representation

LatticeBART: Lattice-to-Lattice Pre-Training for Speech Recognition

Neural Lattice Search for Speech Recognition.

Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models

Exploiting Future Word Contexts in Neural Network Language Models for Speech Recognition.

Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks

Improved lattice-based spoken document retrieval by directly learning from the evaluation measures

Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers

Lattice-based lightly-supervised acoustic model training

LERT: A Linguistically-motivated Pre-trained Language Model

Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models

Lattice Transformer for Speech Translation

Speech Recognition Lattice-Generating Algorithm with Forward-Backward Language Model

Bi-Lattice LSTM Model with Self-Attention for Chinese NER

Demonstrative Instruction Following in Multimodal LLMs Via Integrating Low-Rank Adaptation with Ensemble Learning

Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation

Chinese NER Using Lattice LSTM

Using Large Language Model for End-to-End Chinese ASR and NER

A Novel LSTM-Based Speech Preprocessor for Speaker Diarization in Realistic Mismatch Conditions.