Abstract:Interpretable multi-hop reading comprehension (RC) over multiple documents is a challenging problem because it demands reasoning over multiple information sources and explaining the answer prediction by providing supporting evidences. In this paper, we propose an effective and interpretable Select, Answer and Explain (SAE) system to solve the multi-document RC problem. Our system first filters out answer-unrelated documents and thus reduce the amount of distraction information. This is achieved by a document classifier trained with a novel pairwise learning-to-rank loss. The selected answer-related documents are then input to a model to jointly predict the answer and supporting sentences. The model is optimized with a multi-task learning objective on both token level for answer prediction and sentence level for supporting sentences prediction, together with an attention-based interaction between these two tasks. Evaluated on HotpotQA, a challenging multi-hop RC data set, the proposed SAE system achieves top competitive performance in distractor setting compared to other existing systems on the leaderboard.
What problem does this paper attempt to address?
This paper proposes a solution to the problem of multi-document interpretable multi-hop reading comprehension. Existing machine reading comprehension systems mainly focus on single-document question answering, while multi-document question answering requires reasoning and explanation of the answer based on multiple information sources, which is a challenging task. To address this, the paper introduces a system called "Selection, Answering, and Explanation" (SAE).
The SAE system first filters out irrelevant documents using a document classifier to reduce interference. This classifier uses a novel dual learning ranking loss function to improve accuracy. Then, the selected relevant documents are input into a model for joint prediction of the answer and supporting sentences. The model is optimized through multi-task learning, focusing on both token-level answer prediction and sentence-level supporting sentence prediction, with attention mechanism for interaction between the two tasks.
Evaluation on the HotpotQA dataset shows that the SAE system is competitive compared to existing systems. However, the current methods have several limitations, including the lack of answer explanation, direct processing of all documents, and reliance on entities. To overcome these limitations, the SAE system integrates document selection, answer prediction, and explanation, enhancing the model's explanatory ability and efficiency.
In summary, this paper aims to develop a system that can handle the task of multi-document reading comprehension, which not only accurately identifies answers but also explains the reasoning process and effectively filters out irrelevant information.