Abstract:Semantic interaction modeling is a fundamental technology in natural language understanding that guides models to extract deep semantic information from text. Currently, the attention mechanism is one of the most effective techniques in semantic interaction modeling, which learns word-level attention representation by measuring the relevance between different words. However, the attention mechanism is limited to word-level semantic interaction, it cannot meet the needs of fine-grained interactive information for some text classification tasks. In recent years, quantum-inspired language modeling methods have successfully constructed quantized representations of language systems in Hilbert spaces, which use density matrices to achieve fine-grained semantic interaction modeling. This paper presents a Q uantum-inspired hierarchical S emantic I nteraction M odel (QSIM), QSIM ), which follows the sememe-word-sentence language construction principle and utilizes quantum entanglement theory to capture hierarchical semantic interaction information in Hilbert space. Our work builds on the idea of the attention mechanism and extends it. Specifically, we explore the original semantic space from a quantum theory perspective and derive the core semantic space using the Schmidt decomposition technique, where: (1) Sememe is represented as the unit vector in the two-dimensional minimum semantic space; (2) Word is represented as reduced density matrices in the core semantic space, where Schmidt coefficients quantify sememe-level semantic interaction. Compared to density matrices, reduced density matrices capture fine-grained semantic interaction information with lower computational cost; (3) Sentence is represented as quantum superposition states of words, and the degree of word-level semantic interaction is measured using entanglement entropy. To evaluate the model's performance, we conducted experiments on 15 text classification datasets. The experimental results demonstrate that our model is superior to classical neural network models and traditional quantum-inspired language models. Furthermore, the experiment also confirms two distinct advantages of QISM: (1) flexibility, , as it can be integrated into various mainstream neural network text classification architectures; and (2) practicability, , as it alleviates the problem of parameter growth inherent in density matrix calculation in quantum language model.

Bi-Directional LSTM with Quantum Attention Mechanism for Sentence Modeling

A light-weight quantum self-attention model for classical data classification

Quantum Mixed-State Self-Attention Network

Quantum Attention Based Language Model for Answer Selection.

Quantum self-attention neural networks for text classification

A Quantum Many-body Wave Function Inspired Language Modeling Approach

LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory

An attention mechanism and multi-granularity-based Bi-LSTM model for Chinese Q&A system

QSIM: A Quantum-inspired Hierarchical Semantic Interaction Model for Text Classification

QKSAN: A Quantum Kernel Self-Attention Network

A Quantum Expectation Value Based Language Model with Application to Question Answering

A Novel Stochastic LSTM Model Inspired by Quantum Machine Learning

Quantum-inspired Interpretable Deep Learning Architecture for Text Sentiment Analysis

Enhanced Quantum Long Short-Term Memory by Using Bidirectional Ring Variational Quantum Circuit

Bi-directional Long Short-Term Memory Model with Semantic Positional Attention for the Question Answering System

Implementation Guidelines and Innovations in Quantum LSTM Networks

GQHAN: A Grover-inspired Quantum Hard Attention Network

Adapting Pre-trained Language Models for Quantum Natural Language Processing

Quantum Active Learning

Quantum Language Model with Entanglement Embedding for Question Answering