XTQA: Span-Level Explanations of the Textbook Question Answering

Jie Ma,Qi Chai,Jun Liu,Qingyu Yin,Pinghui Wang,Qinghua Zheng
2023-07-24
Abstract:Textbook Question Answering (TQA) is a task that one should answer a diagram/non-diagram question given a large multi-modal context consisting of abundant essays and diagrams. We argue that the explainability of this task should place students as a key aspect to be considered. To address this issue, we devise a novel architecture towards span-level eXplanations of the TQA (XTQA) based on our proposed coarse-to-fine grained algorithm, which can provide not only the answers but also the span-level evidences to choose them for students. This algorithm first coarsely chooses top $M$ paragraphs relevant to questions using the TF-IDF method, and then chooses top $K$ evidence spans finely from all candidate spans within these paragraphs by computing the information gain of each span to questions. Experimental results shows that XTQA significantly improves the state-of-the-art performance compared with baselines. The source code is available at <a class="link-external link-https" href="https://github.com/keep-smile-001/opentqa" rel="external noopener nofollow">this https URL</a>
Computation and Language,Artificial Intelligence
What problem does this paper attempt to address?
The paper is primarily dedicated to addressing the issue of explainability in the Textbook Question Answering (TQA) task. Specifically, the goals of the paper can be summarized as follows: 1. **Propose a new TQA framework (XTQA)**: To address the under-explored issue of providing explanations in the TQA task, the authors designed a new architecture called XTQA. This architecture considers the entire textbook course's textual context as candidate evidence and utilizes the proposed coarse-to-fine Explanation Extraction (EE) algorithm to extract paragraph-level explanations. 2. **Address the explainability issue in a multimodal context**: The TQA task requires the system to automatically answer chart or non-chart questions posed in a multimodal context containing a large amount of text and charts. To deepen human understanding of the learned knowledge, an ideal TQA system should not only provide answers but also explanations. 3. **Propose an Explanation Extraction algorithm (EE)**: The authors proposed an Explanation Extraction algorithm that can extract key information (i.e., paragraph-level explanations) from relevant paragraphs, which is crucial for answering different questions. This algorithm can be applied not only within the XTQA framework but also integrated into other TQA methods to enhance their explainability and performance. 4. **Experimental validation**: Through experiments on the CK12-QA dataset, it was demonstrated that XTQA achieved the best overall results in providing explanations and also performed excellently in TQA performance, achieving the best accuracy. 5. **Summary of contributions**: The main contributions of the paper include the proposal of a new TQA architecture, a new Explanation Extraction algorithm, and extensive experimental validation of the method's effectiveness. In summary, the paper focuses on how to provide reasonable explanations for answers in the TQA task, aiming to deepen human users' understanding of knowledge by improving the system's explainability.