Abstract:Machine Reading Comprehension (MRC) is one of the most challenging tasks in both NLP and IR researches. Recently, a number of deep neural models have been successfully adopted to some simplified MRC task settings, whose performances were close to or even better than human beings. However, these models still have large performance gaps with human beings in more practical settings, such as MS MARCO and DuReader datasets. Although there are many works studying human reading behavior, the behavior patterns in complex reading comprehension scenarios remain under-investigated. We believe that a better understanding of how human reads and allocates their attention during reading comprehension processes can help improve the performance of MRC tasks. In this paper, we conduct a lab study to investigate human's reading behavior patterns during reading comprehension tasks, where 32 users are recruited to take 60 distinct tasks. By analyzing the collected eye-tracking data and answers from participants, we propose a two-stage reading behavior model, in which the first stage is to search for possible answer candidates and the second stage is to generate the final answer through a comparison and verification process. We also find that human's attention distribution is affected by both question-dependent factors (e.g., answer and soft matching signal with questions) and question-independent factors (e.g., position, IDF and Part-of-Speech tags of words). We extract features derived from the two-stage reading behavior model to predict human's attention signals during reading comprehension, which significantly improves performance in the MRC task. Findings in our work may bring insight into the understanding of human reading and information seeking processes, and help the machine to better meet users' information needs.

Effective Strategies for Low-Resource Reading Comprehension

Feeding What You Need by Understanding What You Learned

XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

From Good to Best: Two-Stage Training for Cross-Lingual Machine Reading Comprehension

A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets.

A survey of deep learning techniques for machine reading comprehension

Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond

Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension

A Span-Extraction Dataset for Chinese Machine Reading Comprehension

Multi-Strategy to Improve Reading Comprehension

Improving Low-resource Reading Comprehension via Cross-lingual Transposition Rethinking

Human Behavior Inspired Machine Reading Comprehension

Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasets

Towards Human-level Machine Reading Comprehension: Reasoning and Inference with Multiple Strategies.

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation

Analysis and Improvement of External Knowledge Usage in Machine Multi-Choice Reading Comprehension Tasks

DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications

Machine Reading Comprehension: a Literature Review

Domain Adaptive Multi-Task Transformer for Low-Resource Machine Reading Comprehension

CPCM-MRC: A Few-shot Machine Reading Comprehension Approach for Intelligent Bridge Management via Continual Pre-training and Copy Mechanism

Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension