Abstract:Answer selection is a crucial subtask in the question answering (QA) system. Conventional avenues for this task mainly concentrate on developing linguistic tools that are limited in both performance and practicability. Answer selection approaches based on deep learning have been well investigated with the tremendous success of deep learning in natural language processing. However, the traditional neural networks employed in existing answer selection models, i.e., recursive neural network or convolutional neural network, typically suffer from obtaining the global text information due to their operating mechanisms. The recent Transformer neural network is considered to be good at extracting the global information by employing only self-attention mechanism. Thus, in this paper, we design a Transformer-based neural network for answer selection, where we deploy a bidirectional long short-term memory (BiLSTM) behind the Transformer to acquire both global information and sequential features in the question or answer sentence. Different from the original Transformer, our Transformer-based network focuses on sentence embedding rather than the seq2seq task. In addition, we employ a BiLSTM rather than utilizing the position encoding to incorporate sequential features as the universal Transformer does. Furthermore, we apply three aggregated strategies to generate sentence embeddings for question and answer, i.e., the weighted mean pooling, the max pooling, and the attentive pooling, leading to three corresponding Transformer-based models, i.e., QA-TF WP , QA-TF MP , and QA-TF AP , respectively. Finally, we evaluate our proposals on a popular QA dataset WikiQA. The experimental results demonstrate that our proposed Transformer-based answer selection models can produce a better performance compared with several competitive baselines. In detail, our best model outperforms the state-of-the-art baseline by up to 2.37%, 2.83%, and 3.79% in terms of MAP, MRR, and accuracy, respectively.

Part-Of-Speech And Position Attention Mechanism Based Blstm For Question Answering System

Bi-directional Long Short-Term Memory Model with Semantic Positional Attention for the Question Answering System

An attention mechanism and multi-granularity-based Bi-LSTM model for Chinese Q&A system

A Joint Model For Question-Answering Over Traditional Chinese Medicine

Coattention based BiLSTM for Answer Selection

Attention-based Encoder-Decoder Model for Answer Selection in Question Answering

Medical Answer Selection Based on Two Attention Mechanisms with BiRNN

Enhancing Document-Based Question Answering via Interaction Between Question Words and POS Tags.

Latent Attention Network With Position Perception for Visual Question Answering

Pay More Attention - Neural Architectures for Question-Answering

Hierarchical Multi-dimensional Attention Model for Answer Selection.

Improved Blending Attention Mechanism in Visual Question Answering

A Better Way to Attend: Attention with Trees for Video Question Answering

Transformer-Based Neural Network for Answer Selection in Question Answering

Discriminative Question Answering Via Cascade Prompt Learning and Sentence Level Attention Mechanism.

BamnetTL: Bidirectional Attention Memory Network with Transfer Learning for Question Answering Matching

Bi-Directional Gated Memory Networks For Answer Selection

Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network

Visual Question Answering Based on Position Alignment

Empirical Study on Deep Learning Models for Question Answering.

Attention-based BILSTM network with part-of-speech features for Chinese text classification