Abstract:The knowledge graph, a networked structure designed to organize the vast and heterogeneous knowledge existing in the real world, has gained widespread adoption as a background knowledge base for intelligent systems. Nevertheless, the incompleteness of knowledge graphs has been widely recognized as a significant challenge in their development and application. Recently, multi-hop knowledge graph reasoning has been an attractive method for completing a knowledge graph. The multi-hop knowledge graph reasoning based on the reinforcement learning (RL) framework has achieved promising performance in terms of interpretability and scalability. An RL agent automatically reasons over a KG under the guidance of a policy network. When faced with a query, obtaining the approximate range of the answer first and then delving into individual options is a more efficient approach compared to traversing all candidate answers. However, existing RL-based methods have two limitations. First, they lack the ability to filter candidate decisions, making it challenging to handle entities with a large number of neighbors. Second, they fail to consider the intrinsic correlations between entities. To address these limitations, we propose a novel hierarchical knowledge graph reasoning approach HiKGR , which leverages the concept information of entities. Specifically, HiKGR reconstructs the previous action space in RL into a concept space and an instance space, enabling two policies to alternate reasoning at the concept level and the instance level. Furthermore, we propose hierarchical reward functions for the two-level policies to achieve joint optimization. The hierarchical reasoning approach we propose is capable of selecting more reasonable candidate decisions and optimizing the decision space. Experimental results reveal that HiKGR significantly outperforms existing RL-based methods and drastically reduces the action space size.

Robot Representation and Reasoning with Knowledge from Reinforcement Learning

Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Survey

Reason more like human: Incorporating meta information into hierarchical reinforcement learning for knowledge graph reasoning

Multi-hop Knowledge Reasoning with Deep Reinforcement Learning

A Survey of Knowledge Representation in Service Robotics

From Concept to Instance: Hierarchical Reinforced Knowledge Graph Reasoning

Multi-State-Space Reasoning Reinforcement Learning for Long-Horizon RFID-Based Robotic Searching and Planning Tasks

Combining Model-Based $Q$ -Learning with Structural Knowledge Transfer for Robot Skill Learning

How Far Can I Reach? A Case Study in Robot Knowledge Acquisition

Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics

Knowledge Graph Reasoning with Self-supervised Reinforcement Learning

Enhancing Multi-Hop Knowledge Graph Reasoning through Reward Shaping Techniques

Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning

Heterogeneous Relational Reasoning in Knowledge Graphs with Reinforcement Learning

Interactive Learning of State Representation through Natural Language Instruction and Explanation

Learning to walk with logical embedding for knowledge reasoning

Reinforcement Learning with Actor-Critic for Knowledge Graph Reasoning

On-Robot Bayesian Reinforcement Learning for POMDPs

A Closer Look at Reward Decomposition for High-Level Robotic Explanations

Reasoning Like Human: Hierarchical Reinforcement Learning for Knowledge Graph Reasoning

Knowledge Representation Learning: A Quantitative Review