Abstract:Temporal Knowledge Graph Reasoning (TKGR) is the task of inferring missing facts for incomplete TKGs in complex scenarios (e.g., transductive and inductive settings), which has been gaining increasing attention. Recently, to mitigate dependence on structured connections in TKGs, text-based methods have been developed to utilize rich linguistic information from entity descriptions. However, suffering from the enormous parameters and inflexibility of pre-trained language models, existing text-based methods struggle to balance the textual knowledge and temporal information with computationally expensive purpose-built training strategies. To tap the potential of text-based models for TKGR in various complex scenarios, we propose ChapTER, a Contrastive historical modeling framework with prefix-tuning for TEmporal Reasoning. ChapTER feeds history-contextualized text into the pseudo-Siamese encoders to strike a textual-temporal balance via contrastive estimation between queries and candidates. By introducing virtual time prefix tokens, it applies a prefix-based tuning method to facilitate the frozen PLM capable for TKGR tasks under different settings. We evaluate ChapTER on four transductive and three few-shot inductive TKGR benchmarks, and experimental results demonstrate that ChapTER achieves superior performance compared to competitive baselines with only 0.17% tuned parameters. We conduct thorough analysis to verify the effectiveness, flexibility and efficiency of ChapTER.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the problem of Temporal Knowledge Graph Reasoning (TKGR), particularly the inference of missing facts in incomplete Temporal Knowledge Graphs (TKGs) in complex scenarios such as inductive and semi-inductive settings. Specifically, the paper addresses the following key issues: 1. **Balancing Textual and Temporal Information**: Existing text-based methods often struggle to balance the textual knowledge in Pre-Trained Language Models (PLMs) with temporal information when leveraging the rich linguistic information in entity descriptions. 2. **Handling Large Parameter Sizes and Flexibility Issues**: Due to the large parameter sizes and low flexibility of pre-trained language models, current methods are expensive and complex in terms of training strategies. 3. **Adapting to Tasks in Different Complex Scenarios**: Efficiently integrating historical information into textual knowledge and adapting to various complex scenarios of temporal knowledge graph reasoning within a unified PLM-based framework. To address the above issues, the paper proposes a new framework named ChapTER (Contrastive historical modeling with Prefix-tuning for TEmporal Reasoning). ChapTER employs a contrastive historical modeling approach and utilizes a prefix-tuning strategy to effectively fine-tune the frozen pre-trained language models. This method not only enables temporal knowledge graph reasoning in different scenarios (including inductive and semi-inductive settings) but also significantly reduces the number of parameters that need to be adjusted, requiring only 0.17% of the parameters to achieve good performance. In summary, the main contribution of this paper is the proposal of a new temporal knowledge graph reasoning framework, ChapTER, which effectively integrates historical information and textual knowledge, achieves efficient temporal reasoning in various complex scenarios, and maintains high flexibility and low computational cost.

Deja vu: Contrastive Historical Modeling with Prefix-tuning for Temporal Knowledge Graph Reasoning

Temporal Knowledge Graph Reasoning Based on Evolutional Representation and Contrastive Learning

Local-Global History-aware Contrastive Learning for Temporal Knowledge Graph Reasoning

TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph Completion

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion

TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting

Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction

MetaTKG: Learning Evolutionary Meta-Knowledge for Temporal Knowledge Graph Reasoning

A Unified Temporal Knowledge Graph Reasoning Model Towards Interpolation and Extrapolation

Language Models-enhanced Semantic Topology Representation Learning for Temporal Knowledge Graph Extrapolation

T-GAP: Learning to Walk across Time for Temporal Knowledge Graph Completion

A Simple But Powerful Graph Encoder for Temporal Knowledge Graph Completion

Chain-of-History Reasoning for Temporal Knowledge Graph Forecasting

RotateQVS: Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion

Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning

Temporal knowledge graph question answering via subgraph reasoning

Reinforcement learning with time intervals for temporal knowledge graph reasoning

ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs

TPmod: A Tendency-Guided Prediction Model for Temporal Knowledge Graph Completion

Self-Improvement Programming for Temporal Knowledge Graph Question Answering