Abstract:The knowledge graph, a networked structure designed to organize the vast and heterogeneous knowledge existing in the real world, has gained widespread adoption as a background knowledge base for intelligent systems. Nevertheless, the incompleteness of knowledge graphs has been widely recognized as a significant challenge in their development and application. Recently, multi-hop knowledge graph reasoning has been an attractive method for completing a knowledge graph. The multi-hop knowledge graph reasoning based on the reinforcement learning (RL) framework has achieved promising performance in terms of interpretability and scalability. An RL agent automatically reasons over a KG under the guidance of a policy network. When faced with a query, obtaining the approximate range of the answer first and then delving into individual options is a more efficient approach compared to traversing all candidate answers. However, existing RL-based methods have two limitations. First, they lack the ability to filter candidate decisions, making it challenging to handle entities with a large number of neighbors. Second, they fail to consider the intrinsic correlations between entities. To address these limitations, we propose a novel hierarchical knowledge graph reasoning approach HiKGR , which leverages the concept information of entities. Specifically, HiKGR reconstructs the previous action space in RL into a concept space and an instance space, enabling two policies to alternate reasoning at the concept level and the instance level. Furthermore, we propose hierarchical reward functions for the two-level policies to achieve joint optimization. The hierarchical reasoning approach we propose is capable of selecting more reasonable candidate decisions and optimizing the decision space. Experimental results reveal that HiKGR significantly outperforms existing RL-based methods and drastically reduces the action space size.

ARL: analogical reinforcement learning for knowledge graph reasoning

Neural Axiom Network for Knowledge Graph Reasoning

Attention-based Exploitation and Exploration Strategy for Multi-Hop Knowledge Graph Reasoning

Causal Reinforcement Learning for Knowledge Graph Reasoning

Heterogeneous Relational Reasoning in Knowledge Graphs with Reinforcement Learning

From Concept to Instance: Hierarchical Reinforced Knowledge Graph Reasoning

Step by Step: A Hierarchical Framework for Multi-Hop Knowledge Graph Reasoning with Reinforcement Learning

Attention-Based Relational Graph Convolutional Network for Knowledge Graph Reasoning

Cooperative Reasoning on Knowledge Graph and Corpus: A Multi-agentReinforcement Learning Approach

Adversary and Attention Guided Knowledge Graph Reasoning Based on Reinforcement Learning

Analogical Inference Enhanced Knowledge Graph Embedding

RLAT: Multi-hop temporal knowledge graph reasoning based on reinforcement learning and attention mechanism

HackRL: Reinforcement learning with hierarchical attention for cross-graph knowledge fusion and collaborative reasoning

Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning

Reasoning Like Human: Hierarchical Reinforcement Learning for Knowledge Graph Reasoning

Multi-hop Knowledge Reasoning with Deep Reinforcement Learning

Learning to Walk with Dual Agents for Knowledge Graph Reasoning

Dynamic knowledge graph reasoning based on deep reinforcement learning

Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning

ExKGR: Explainable Multi-hop Reasoning for Evolving Knowledge Graph

Hierarchical Policy Network with Multi-agent for Knowledge Graph Reasoning Based on Reinforcement Learning