Abstract:Machine comprehension is a representative task of natural language understanding. Typically, we are given context paragraph and the objective is to answer a question that depends on the context. Such a problem requires to model the complex interactions between the context paragraph and the question. Lately, attention mechanisms have been found to be quite successful at these tasks and in particular, attention mechanisms with attention flow from both context-to-question and question-to-context have been proven to be quite useful. In this paper, we study two state-of-the-art attention mechanisms called Bi-Directional Attention Flow (BiDAF) and Dynamic Co-Attention Network (DCN) and propose a hybrid scheme combining these two architectures that gives better overall performance. Moreover, we also suggest a new simpler attention mechanism that we call Double Cross Attention (DCA) that provides better results compared to both BiDAF and Co-Attention mechanisms while providing similar performance as the hybrid scheme. The objective of our paper is to focus particularly on the attention layer and to suggest improvements on that. Our experimental evaluations show that both our proposed models achieve superior results on the Stanford Question Answering Dataset (SQuAD) compared to BiDAF and DCN attention mechanisms.

Attention-Based Memory Network for Sentence-Level Question Answering.

Empirical Study on Deep Learning Models for Question Answering.

Skip-Thought Memory Networks

Long-Term Memory Networks for Question Answering

A Question-Focused Multi-Factor Attention Network for Question Answering

Question-Aware Memory Network for Multi-hop Question Answering in Human-Robot Interaction

Bi-directional Long Short-Term Memory Model with Semantic Positional Attention for the Question Answering System

MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension.

Memory Augmented Deep Recurrent Neural Network for Video Question Answering

Attentive Interactive Neural Networks for Answer Selection in Community Question Answering.

Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering.

Feature Augmented Memory with Global Attention Network for VideoQA

AMQAN: Adaptive Multi-Attention Question-Answer Networks for Answer Selection

Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs

A Multi-View Fusion Neural Network for Answer Selection

CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering

MCQA: Multimodal Co-attention Based Network for Question Answering

MemexQA: Visual Memex Question Answering

Visual Question Answering with Memory-Augmented Networks

Pay More Attention - Neural Architectures for Question-Answering

Answering with Memory-Augmented Networks