Abstract:The knowledge graph, a networked structure designed to organize the vast and heterogeneous knowledge existing in the real world, has gained widespread adoption as a background knowledge base for intelligent systems. Nevertheless, the incompleteness of knowledge graphs has been widely recognized as a significant challenge in their development and application. Recently, multi-hop knowledge graph reasoning has been an attractive method for completing a knowledge graph. The multi-hop knowledge graph reasoning based on the reinforcement learning (RL) framework has achieved promising performance in terms of interpretability and scalability. An RL agent automatically reasons over a KG under the guidance of a policy network. When faced with a query, obtaining the approximate range of the answer first and then delving into individual options is a more efficient approach compared to traversing all candidate answers. However, existing RL-based methods have two limitations. First, they lack the ability to filter candidate decisions, making it challenging to handle entities with a large number of neighbors. Second, they fail to consider the intrinsic correlations between entities. To address these limitations, we propose a novel hierarchical knowledge graph reasoning approach HiKGR , which leverages the concept information of entities. Specifically, HiKGR reconstructs the previous action space in RL into a concept space and an instance space, enabling two policies to alternate reasoning at the concept level and the instance level. Furthermore, we propose hierarchical reward functions for the two-level policies to achieve joint optimization. The hierarchical reasoning approach we propose is capable of selecting more reasonable candidate decisions and optimizing the decision space. Experimental results reveal that HiKGR significantly outperforms existing RL-based methods and drastically reduces the action space size.

Path-Based Knowledge Graph Completion Combining Reinforcement Learning with Soft Rules

Knowledge Reasoning via Jointly Modeling Knowledge Graphs and Soft Rules

Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning

Causal Reinforcement Learning for Knowledge Graph Reasoning

Reasoning Like Human: Hierarchical Reinforcement Learning for Knowledge Graph Reasoning

Heterogeneous Relational Reasoning in Knowledge Graphs with Reinforcement Learning

Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning

Learning to walk with logical embedding for knowledge reasoning

Enhancing Path Information with Reinforcement Learning for Few-shot Knowledge Graph Completion.

Dynamic knowledge graph reasoning based on deep reinforcement learning

Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning.

Improving Knowledge Graph Completion Using Soft Rules and Adversarial Learning

Multi-hop Knowledge Reasoning with Deep Reinforcement Learning

Learning and Deduction of Rules for Knowledge Graph Completion

Attention-based Exploitation and Exploration Strategy for Multi-Hop Knowledge Graph Reasoning

Learning to Walk with Dual Agents for Knowledge Graph Reasoning

Multi-hop reasoning over paths in temporal knowledge graphs using reinforcement learning

Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning

Multi-Hop Knowledge Graph Reasoning with Reward Shaping

Step by Step: A Hierarchical Framework for Multi-Hop Knowledge Graph Reasoning with Reinforcement Learning

From Concept to Instance: Hierarchical Reinforced Knowledge Graph Reasoning