Abstract:Semantic interaction modeling is a fundamental technology in natural language understanding that guides models to extract deep semantic information from text. Currently, the attention mechanism is one of the most effective techniques in semantic interaction modeling, which learns word-level attention representation by measuring the relevance between different words. However, the attention mechanism is limited to word-level semantic interaction, it cannot meet the needs of fine-grained interactive information for some text classification tasks. In recent years, quantum-inspired language modeling methods have successfully constructed quantized representations of language systems in Hilbert spaces, which use density matrices to achieve fine-grained semantic interaction modeling. This paper presents a Q uantum-inspired hierarchical S emantic I nteraction M odel (QSIM), QSIM ), which follows the sememe-word-sentence language construction principle and utilizes quantum entanglement theory to capture hierarchical semantic interaction information in Hilbert space. Our work builds on the idea of the attention mechanism and extends it. Specifically, we explore the original semantic space from a quantum theory perspective and derive the core semantic space using the Schmidt decomposition technique, where: (1) Sememe is represented as the unit vector in the two-dimensional minimum semantic space; (2) Word is represented as reduced density matrices in the core semantic space, where Schmidt coefficients quantify sememe-level semantic interaction. Compared to density matrices, reduced density matrices capture fine-grained semantic interaction information with lower computational cost; (3) Sentence is represented as quantum superposition states of words, and the degree of word-level semantic interaction is measured using entanglement entropy. To evaluate the model's performance, we conducted experiments on 15 text classification datasets. The experimental results demonstrate that our model is superior to classical neural network models and traditional quantum-inspired language models. Furthermore, the experiment also confirms two distinct advantages of QISM: (1) flexibility, , as it can be integrated into various mainstream neural network text classification architectures; and (2) practicability, , as it alleviates the problem of parameter growth inherent in density matrix calculation in quantum language model.

A Quantum Expectation Value Based Language Model with Application to Question Answering

Quantum Attention Based Language Model for Answer Selection.

Quantum Language Model with Entanglement Embedding for Question Answering

An Adaptive Contextual Quantum Language Model

A Quantum Many-body Wave Function Inspired Language Modeling Approach

Overcoming Language Priors In Vqa Via Decomposed Linguistic Representations

A Measurement-Based Quantum-Like Language Model for Text Matching.

Modeling Quantum Entanglements in Quantum Language Models

A Survey of Quantum Language Models

Complex-valued Neural Network-based Quantum Language Models

QSIM: A Quantum-inspired Hierarchical Semantic Interaction Model for Text Classification

Quantum-Inspired Neural Language Representation, Matching and Understanding.

Modeling Relevance Judgement Inspired by Quantum Weak Measurement

Context-aware Multi-level Question Embedding Fusion for visual question answering

QLSC: A Query Latent Semantic Calibrator for Robust Extractive Question Answering

Modeling Semantic Inspired by Quantum Composite System

A Retrieval Model for Question in Community Question Answering System.

Quantum-Inspired Fusion for Open-Domain Question Answering

A Quantum Query Expansion Approach for Session Search.

Quantum-Like Contextuality in Large Language Models

Mining Latent Semantic Correlation Inspired by Quantum Entanglement