Abstract:Knowledge Graph (KG) is identified as a major area in artificial intelligence, which is used for many real-world applications. The task of knowledge graph reasoning has been widely used and proven to be effective, which aims to find these reasonable paths for various relations to solve the issue of incompleteness in KGs. However, many previous works on KG reasoning, such as path-based or reinforcement learning-based methods, are too reliant on the pre-training, where the paths from the head entity and the target entity must be given to pre-train the model, which would easily lead the model to overfit on the given paths seen in the pre-training. To address this issue, we propose a novel reasoning model named MemoryPath with a deep reinforcement learning framework, which incorporates Long Short Term Memory (LSTM) and graph attention mechanism to form the memory component. The well-designed memory component can get rid of the pre-training so that the model doesn’t depend on the given target entity for training. A tailored mechanism of reinforcement learning is presented in this proposed deep reinforcement framework to optimize the training procedure, where two metrics, Mean Selection Rate (MSR) and Mean Alternative Rate (MAR), are defined to quantitatively measure the complexities of the query relations. Meanwhile, three different training mechanisms, Action Dropout, Reward Shaping and Force Forward, are proposed to optimize the training process of the proposed MemoryPath. The proposed MemoryPath is validated on two datasets from FB15K-237 and NELL-995 on different tasks including fact prediction, link prediction and success rate in finding paths. The experimental results demonstrate that the tailored mechanism of reinforcement learning make the MemoryPath achieves state-of-the-art performance comparing with the other models. Also, the qualitative analysis indicates that the MemoryPath can store the learning process and automatically find the promising paths for a reasoning task during the training, and shows the effectiveness of the memory component.

Learning to Rehearse in Long Sequence Memorization

Continual Memory: Can We Reason after Long-Term Memorization?

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Learning to Update Auto-associative Memory in Recurrent Neural Networks for Improving Sequence Memorization.

Multi-Domain Multi-Task Rehearsal for Lifelong Learning

A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information

Long short-term enhanced memory for sequential recommendation

Learning to Reason and Memorize with Self-Notes

MemoryPath: A Deep Reinforcement Learning Framework for Incorporating Memory Component into Knowledge Graph Reasoning

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal

Ordered Memory.

On Memorization of Large Language Models in Logical Reasoning

RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Saliency-Guided Hidden Associative Replay for Continual Learning

Pseudo-rehearsal: Achieving deep reinforcement learning without catastrophic forgetting

ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigation

Relational recurrent neural networks

Fast-backward Replay of Sequentially Memorized Items in Humans.

LaMemo: Language Modeling with Look-Ahead Memory

Learning Human Cognitive Appraisal Through Reinforcement Memory Unit

Replay-enhanced Continual Reinforcement Learning