Abstract:Despite recent advancements, like Large Language Models (LLMs), in Question Answering with Machine Reading (QAMR), improving the factuality and faithfulness of QAMR models remains a significant challenge. QAMR models require both language knowledge and world knowledge to answer questions. Language knowledge encompasses syntax, semantics, pragmatics, and other language-specific elements. The extent of language knowledge reflects the model's language understanding capabilities. World knowledge, which refers to people's cognition of the world, may be parameterized knowledge of the pre-trained language models or textual knowledge of passages. We conduct a comparative study on these two kinds of knowledge and find that language knowledge is stable, while only part of world knowledge is stable and reliable. This motivates us to utilize textual knowledge of passages and avoid parameterized unstable world knowledge of pre-trained language models for QAMR task. To this end, this paper introduces the concept of Answerable without relying on unstable world knowledge external to the passage (AUKE) to determine whether a question can be answered without using parameterized unstable world knowledge of pre-trained language models. . We then define evidence as the simplest substring in the passage that supports AUKE. Based on evidence, , we introduce a novel faithfulness metric for the QAMR task. We propose a methodology that combines automated processes with manual refinement to augment QAMR datasets with evidence annotations to facilitate faithfulness evaluations. We apply this method to the Chinese QAMR dataset CMRC 2018 and DRCD to extend two datasets that support evidence-based faithfulness evaluation, CMRCFF (CMRC with Faithfulness) and DRCDFF (CMRC with Faithfulness). To alleviate the potential factuality and faithfulness issues induced by unstable world knowledge, we propose a method called Maximizing Discrimination Masking (MDM), which masks the word with the highest degree of distinguishability. MDM is an approximation method designed to circumvent the reliance on parameterized unstable world knowledge embedded within pre-trained language models utilized by QAMR systems. We conduct experiments under the fine-tune setting and few-shot setting on CMRCFF and DRCDFF. The results verify that our MDM approach can effectively improve the factuality and faithfulness of the models.

An Interpretable Model Using Evidence Information for Multi-Hop Question Answering Over Long Texts

Multi-grained Evidence Inference for Multi-choice Reading Comprehension

Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering

Evidence Sentence Extraction for Machine Reading Comprehension

Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering

Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering

Multi-level Retrieval with Semantic Axiomatic Fuzzy Set Clustering for Question Answering

Maximizing Discrimination Masking for Faithful Question Answering with Machine Reading

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension.

Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable

An Entailment Tree Generation Approach for Multimodal Multi-Hop Question Answering with Mixture-of-Experts and Iterative Feedback Mechanism

Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents

Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation

APER: AdaPtive Evidence-driven Reasoning Network for machine reading comprehension with unanswerable questions

RRQA: Reconfirmed Reader for Open-Domain Question Answering

ExpMRC: Explainability Evaluation for Machine Reading Comprehension

Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering

Multiple Perspective Answer Reranking for Multi-passage Reading Comprehension.

REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training

MGRC: an End-to-End Multigranularity Reading Comprehension Model for Question Answering