Abstract:Answer selection is a crucial subtask in the question answering (QA) system. Conventional avenues for this task mainly concentrate on developing linguistic tools that are limited in both performance and practicability. Answer selection approaches based on deep learning have been well investigated with the tremendous success of deep learning in natural language processing. However, the traditional neural networks employed in existing answer selection models, i.e., recursive neural network or convolutional neural network, typically suffer from obtaining the global text information due to their operating mechanisms. The recent Transformer neural network is considered to be good at extracting the global information by employing only self-attention mechanism. Thus, in this paper, we design a Transformer-based neural network for answer selection, where we deploy a bidirectional long short-term memory (BiLSTM) behind the Transformer to acquire both global information and sequential features in the question or answer sentence. Different from the original Transformer, our Transformer-based network focuses on sentence embedding rather than the seq2seq task. In addition, we employ a BiLSTM rather than utilizing the position encoding to incorporate sequential features as the universal Transformer does. Furthermore, we apply three aggregated strategies to generate sentence embeddings for question and answer, i.e., the weighted mean pooling, the max pooling, and the attentive pooling, leading to three corresponding Transformer-based models, i.e., QA-TF WP , QA-TF MP , and QA-TF AP , respectively. Finally, we evaluate our proposals on a popular QA dataset WikiQA. The experimental results demonstrate that our proposed Transformer-based answer selection models can produce a better performance compared with several competitive baselines. In detail, our best model outperforms the state-of-the-art baseline by up to 2.37%, 2.83%, and 3.79% in terms of MAP, MRR, and accuracy, respectively.

Local-to-Global Structure-Aware Transformer for Question Answering over Structured Knowledge.

Table Fact Verification with Structure-Aware Transformer

Structural analysis of an all-purpose question answering model

TRAR: Routing the Attention Spans in Transformer for Visual Question Answering

Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation

TransKGQA: Enhanced Knowledge Graph Question Answering With Sentence Transformers

Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer

Cluster-Former: Clustering-based Sparse Transformer for Question Answering.

Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables

Multimodal Graph Transformer for Multimodal Question Answering

Structure-aware Pre-training for Table Understanding with Tree-based Transformers.

Positional Attention Guided Transformer-Like Architecture for Visual Question Answering

Structure-aware Pre-training for Table Understanding with Tree-based Transformers

Transformer-Based Neural Network for Answer Selection in Question Answering

Modeling Graph Structure in Transformer for Better AMR-to-Text Generation.

TLNet: Temporal Span Localization Network with Collaborative Graph Reasoning for Video Question Answering

Graph Reasoning Transformers for Knowledge-Aware Question Answering

MS-Transformer: Introduce multiple structural priors into a unified transformer for encoding sentences

Syntax-informed Question Answering with Heterogeneous Graph Transformer

Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA

TUTA: Tree-based Transformers for Generally Structured Table Pre-training