Abstract:Language modeling is essential in Natural Language Processing and Information Retrieval related tasks. After the statistical language models, Quantum Language Model (QLM) has been proposed to unify both single words and compound terms in the same probability space without extending term space exponentially. Although QLM achieved good performance in ad hoc retrieval, it still has two major limitations: (1) QLM cannot make use of supervised information, mainly due to the iterative and non-differentiable estimation of the density matrix, which represents both queries and documents in QLM. (2) QLM assumes the exchangeability of words or word dependencies, neglecting the order or position information of words. This article aims to generalize QLM and make it applicable to more complicated matching tasks (e.g., Question Answering) beyond ad hoc retrieval. We propose a complex-valued neural network-based QLM solution called C-NNQLM to employ an end-to-end approach to build and train density matrices in a light-weight and differentiable manner, and it can therefore make use of external well-trained word vectors and supervised labels. Furthermore, C-NNQLM adopts complex-valued word vectors whose phase vectors can directly encode the order (or position) information of words. Note that complex numbers are also essential in the quantum theory. We show that the real-valued NNQLM (R-NNQLM) is a special case of C-NNQLM. The experimental results on the QA task show that both R-NNQLM and C-NNQLM achieve much better performance than the vanilla QLM, and C-NNQLM’s performance is on par with state-of-the-art neural network models. We also evaluate the proposed C-NNQLM on text classification and document retrieval tasks. The results on most datasets show that the C-NNQLM can outperform R-NNQLM, which demonstrates the usefulness of the complex representation for words and sentences in C-NNQLM.

Complex Rotation Quantum Dynamic Neural Networks (CRQDNN) using Complex Quantum Neuron (CQN): Applications to time series prediction.

Predictive Modelling of Quantum Process with Neural Networks

Quantum recurrent neural networks for sequential learning

CTRQNets & LQNets: Continuous Time Recurrent and Liquid Quantum Neural Networks

Quantum-inspired Complex Convolutional Neural Networks

QFCNN: Quantum Fourier Convolutional Neural Network

EQNN: Enhanced Quantum Neural Network

Quantum-Discrete-Map-Based Recurrent Neural Networks

Quantum State Classification Via Complex-Valued Neural Networks

Quantum neural networks with deep residual learning

Quantum Fourier Convolutional Network

Quantum Neural Network for Quantum Neural Computing

Quantum Neural Network and Its Applications

A Quantum-Implementable Neural Network Model

Complex-valued Neural Network-based Quantum Language Models

A Classic-Quantum Hybrid Network Framework: CQH-Net

A Quantum Neural Network Based On H Epsilon Q-Gate And Its Application

Recurrent Quantum Neural Networks

On the Features of Quantum Neuron

Coherent Feed Forward Quantum Neural Network