Abstract:Textual representations play an important role in the field of natural language processing (NLP). The efficiency of NLP tasks, such as text comprehension and information extraction, can be significantly improved with proper textual representations. As neural networks are gradually applied to learn the representation of words and phrases, fairly efficient models of learning short text representations have been developed, such as the continuous bag of words (CBOW) and skip-gram models, and they have been extensively employed in a variety of NLP tasks. Because of the complex structure generated by the longer text lengths, such as sentences, algorithms appropriate for learning short textual representations are not applicable for learning long textual representations. One method of learning long textual representations is the Long Short-Term Memory (LSTM) network, which is suitable for processing sequences. However, the standard LSTM does not adequately address the primary sentence structure (subject, predicate and object), which is an important factor for producing appropriate sentence representations. To resolve this issue, this paper proposes the dependency-based LSTM model (D-LSTM). The D-LSTM divides a sentence representation into two parts: a basic component and a supporting component. The D-LSTM uses a pre-trained dependency parser to obtain the primary sentence information and generate supporting components, and it also uses a standard LSTM model to generate the basic sentence components. A weight factor that can adjust the ratio of the basic and supporting components in a sentence is introduced to generate the sentence representation. Compared with the representation learned by the standard LSTM, the sentence representation learned by the D-LSTM contains a greater amount of useful information. The experimental results show that the D-LSTM is superior to the standard LSTM for sentences involving compositional knowledge (SICK) data.

Semantic Dependency and Local Convolution for Enhancing Naturalness and Tone in Text-to-speech Synthesis

Dependency-based Siamese Long Short-Term Memory Network for Learning Sentence Representations.

Enhancing Local Dependencies for Transformer-Based Text-to-Speech via Hybrid Lightweight Convolution

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Dependency Parsing based Semantic Representation Learning with Graph Neural Network for Enhancing Expressiveness of Text-to-Speech

Boosting Neural Machine Translation with Dependency-Scaled Self-Attention Network.

Deps-SAN: Neural Machine Translation with Dependency-Scaled Self-Attention Network

IMPROVING NATURALNESS AND CONTROLLABILITY OF SEQUENCE-TO-SEQUENCE SPEECH SYNTHESIS BY LEARNING LOCAL PROSODY REPRESENTATIONS

Mandarin Text-to-Speech Front-End with Lightweight Distilled Convolution Network

A Local Information Perception Enhancement–Based Method for Chinese NER

Neural Speech Synthesis with Transformer Network.

Dependency-based Convolutional Neural Networks for Sentence Embedding

Prosodic Structure Prediction Using Deep Self-attention Neural Network

Dependency-Based Local Attention Approach to Neural Machine Translation

Monaural Speech Enhancement with Deep Residual-Dense Lattice Network and Attention Mechanism in the Time Domain

An improved wav2vec 2.0 pre-training approach using enhanced local dependency modeling for speech recognition

Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding

Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

SETransformer: Speech Enhancement Transformer

A speech enhancement model based on noise component decomposition: Inspired by human cognitive behavior