Abstract:Semantic interaction modeling is a fundamental technology in natural language understanding that guides models to extract deep semantic information from text. Currently, the attention mechanism is one of the most effective techniques in semantic interaction modeling, which learns word-level attention representation by measuring the relevance between different words. However, the attention mechanism is limited to word-level semantic interaction, it cannot meet the needs of fine-grained interactive information for some text classification tasks. In recent years, quantum-inspired language modeling methods have successfully constructed quantized representations of language systems in Hilbert spaces, which use density matrices to achieve fine-grained semantic interaction modeling. This paper presents a Q uantum-inspired hierarchical S emantic I nteraction M odel (QSIM), QSIM ), which follows the sememe-word-sentence language construction principle and utilizes quantum entanglement theory to capture hierarchical semantic interaction information in Hilbert space. Our work builds on the idea of the attention mechanism and extends it. Specifically, we explore the original semantic space from a quantum theory perspective and derive the core semantic space using the Schmidt decomposition technique, where: (1) Sememe is represented as the unit vector in the two-dimensional minimum semantic space; (2) Word is represented as reduced density matrices in the core semantic space, where Schmidt coefficients quantify sememe-level semantic interaction. Compared to density matrices, reduced density matrices capture fine-grained semantic interaction information with lower computational cost; (3) Sentence is represented as quantum superposition states of words, and the degree of word-level semantic interaction is measured using entanglement entropy. To evaluate the model's performance, we conducted experiments on 15 text classification datasets. The experimental results demonstrate that our model is superior to classical neural network models and traditional quantum-inspired language models. Furthermore, the experiment also confirms two distinct advantages of QISM: (1) flexibility, , as it can be integrated into various mainstream neural network text classification architectures; and (2) practicability, , as it alleviates the problem of parameter growth inherent in density matrix calculation in quantum language model.

Quantum-Inspired DMATT-BiGRU for Conversational Sentiment Analysis

Quantum-inspired Interpretable Deep Learning Architecture for Text Sentiment Analysis

An Effective Strategy for Sentiment Analysis Based on Complex-Valued Embedding and Quantum Long Short-Term Memory Neural Network

ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis

AB-GRU: An attention-based bidirectional GRU model for multimodal sentiment fusion and analysis

Utterance-Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM.

A Survey of Quantum-Cognitively Inspired Sentiment Analysis Models

TDAM: a Topic-Dependent Attention Model for Sentiment Analysis

Applying QNLP to sentiment analysis in finance

QSIM: A Quantum-inspired Hierarchical Semantic Interaction Model for Text Classification

Quantum Mixed-State Self-Attention Network

Position-Enhanced Multi-Head Self-Attention Based Bidirectional Gated Recurrent Unit for Aspect-Level Sentiment Classification

Convolution-Based Neural Attention with Applications to Sentiment Classification

DiaASQ : A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis

Transformer-Encoder-GRU (T-E-GRU) for Chinese Sentiment Analysis on Chinese Comment Text

Hierarchical graph contrastive learning framework based on quantum neural networks for sentiment analysis

Quantum self-attention neural networks for text classification

Attention-Based CNN and Bi-LSTM Model Based on TF-IDF and GloVe Word Embedding for Sentiment Analysis

Quantum Attention Based Language Model for Answer Selection.

Multimodal sentiment analysis based on multi-head attention mechanism