Multi-hop Question Answering under Temporal Knowledge Editing

Keyuan Cheng,Gang Lin,Haoyang Fei,Yuxuan zhai,Lu Yu,Muhammad Asif Ali,Lijie Hu,Di Wang

2024-03-31

Abstract:Multi-hop question answering (MQA) under knowledge editing (KE) has garnered significant attention in the era of large language models. However, existing models for MQA under KE exhibit poor performance when dealing with questions containing explicit temporal contexts. To address this limitation, we propose a novel framework, namely TEMPoral knowLEdge augmented Multi-hop Question Answering (TEMPLE-MQA). Unlike previous methods, TEMPLE-MQA first constructs a time-aware graph (TAG) to store edit knowledge in a structured manner. Then, through our proposed inference path, structural retrieval, and joint reasoning stages, TEMPLE-MQA effectively discerns temporal contexts within the question query. Experiments on benchmark datasets demonstrate that TEMPLE-MQA significantly outperforms baseline models. Additionally, we contribute a new dataset, namely TKEMQA, which serves as the inaugural benchmark tailored specifically for MQA with temporal scopes.

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problem of how to effectively handle explicit temporal context in multi - hop question answering (MQA) in large - scale language models. Specifically, existing multi - hop question answering models perform poorly when dealing with questions with clear temporal backgrounds, especially in the case of knowledge editing (KE). For example, when updating a certain fact (such as the change of the US president from Trump to Biden), the related chain reactions (such as the change of the First Lady of the United States) need to be accurately captured and updated. However, existing methods often fail to accurately handle such problems due to the lack of effective management of temporal information.

To overcome these limitations, the paper proposes a new framework - **TEMPLE - MQA** (Temporal Knowledge Augmented Multi - hop Question Answering). The main contributions of **TEMPLE - MQA** include:

1. **Constructing a Temporal - Aware Graph (TAG)**: Different from existing methods, **TEMPLE - MQA** first constructs a temporal - aware graph (TAG) to store editing knowledge in a structured way. This helps to more effectively identify and utilize temporal context information when dealing with multi - hop question answering.

2. **Improved Inference Paths and Joint Inference Mechanisms**: **TEMPLE - MQA** can handle multi - hop questions more accurately through the designed inference paths and joint inference mechanisms. Specifically, it uses pre - trained language models to generate inference paths and combines TAG for step - by - step joint inference to obtain the final answer.

3. **Data Augmentation and Context - Dependent Concept Filtering**: To improve retrieval performance, **TEMPLE - MQA** adopts data augmentation techniques to capture aliases of entity names, helping to solve the entity disambiguation problem. At the same time, it also uses context - dependent concepts to explicitly filter edits, further improving the accuracy of retrieval.

4. **Experimental Verification**: Experimental results show that **TEMPLE - MQA** significantly outperforms existing baseline models on multiple benchmark datasets. In addition, the authors also contribute a new dataset **TKEMQA**, which is specifically used to evaluate multi - hop question answering tasks with a time range.

In conclusion, this paper aims to improve the performance of large - scale language models in handling multi - hop question answering tasks with temporal backgrounds by introducing temporal - aware graphs and improved inference mechanisms.