Quantum Attention Based Language Model for Answer Selection.

Qin Zhao,Chenguang Hou,Ruifeng Xu
DOI: https://doi.org/10.1007/978-3-030-96033-9_4
2021-01-01
Abstract:Attention mechanism originally introduced for machine translation has a wide application in NLP tasks. By attending more important data with higher wight, the mechanism has the potential to improve neural networks’ performance. Meanwhile, along with the pursuit of higher performance is the challenge of neural networks’ interpretability. Based on quantum probability theory, quantum language model is just such an attempt and has drawn increasing attention. In this paper, we intend to investigate a balance between model’s performance and interpretability, and propose a quantum attention based language model. Density matrix which carries the appearance probability of any word is used to construct quantum attention. Applied in a typical Question Answering task—Answer Selection, our model shows an effective performance on TREC-QA and WIKI-QA datasets.
What problem does this paper attempt to address?