Abstract:Knowledge Graph (KG) is identified as a major area in artificial intelligence, which is used for many real-world applications. The task of knowledge graph reasoning has been widely used and proven to be effective, which aims to find these reasonable paths for various relations to solve the issue of incompleteness in KGs. However, many previous works on KG reasoning, such as path-based or reinforcement learning-based methods, are too reliant on the pre-training, where the paths from the head entity and the target entity must be given to pre-train the model, which would easily lead the model to overfit on the given paths seen in the pre-training. To address this issue, we propose a novel reasoning model named MemoryPath with a deep reinforcement learning framework, which incorporates Long Short Term Memory (LSTM) and graph attention mechanism to form the memory component. The well-designed memory component can get rid of the pre-training so that the model doesn’t depend on the given target entity for training. A tailored mechanism of reinforcement learning is presented in this proposed deep reinforcement framework to optimize the training procedure, where two metrics, Mean Selection Rate (MSR) and Mean Alternative Rate (MAR), are defined to quantitatively measure the complexities of the query relations. Meanwhile, three different training mechanisms, Action Dropout, Reward Shaping and Force Forward, are proposed to optimize the training process of the proposed MemoryPath. The proposed MemoryPath is validated on two datasets from FB15K-237 and NELL-995 on different tasks including fact prediction, link prediction and success rate in finding paths. The experimental results demonstrate that the tailored mechanism of reinforcement learning make the MemoryPath achieves state-of-the-art performance comparing with the other models. Also, the qualitative analysis indicates that the MemoryPath can store the learning process and automatically find the promising paths for a reasoning task during the training, and shows the effectiveness of the memory component.

Sample Efficient Reinforcement Learning Using Graph-Based Memory Reconstruction.

Sample Efficient Reinforcement Learning Method Via High Efficient Episodic Memory.

Episodic Reinforcement Learning with Associative Memory.

Sample-efficient multi-agent reinforcement learning with masked reconstruction

AdaMemento: Adaptive Memory-Assisted Policy Optimization for Reinforcement Learning

Sample-efficient Deep Reinforcement Learning with Directed Associative Graph

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Model-Based Reinforcement Learning Via Imagination with Derived Memory.

Remember the Past for Better Future: Memory-Augmented Offline RL

MemoryPath: A Deep Reinforcement Learning Framework for Incorporating Memory Component into Knowledge Graph Reasoning

Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks

Prioritized Generative Replay

Augmented Replay Memory in Reinforcement Learning With Continuous Control

Learning a World Model With Multitimescale Memory Augmentation

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

Deep Reinforcement Learning with Parametric Episodic Memory

Brain Inspired Episodic Memory Deep Q-Networks for Sparse Reward

Deep reinforcement learning via good choice resampling experience replay memory

MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards

Integrating human learning and reinforcement learning: A novel approach to agent training

Reinforcement Learning with Fast and Forgetful Memory