Large Scale Semantic Rule-based Backward Chaining Reasoning on Spark

Rong GU,Shanyong WANG,Chen GUO,Chunfeng YUAN,Yihua HUANG

DOI: https://doi.org/10.3969/j.issn.1003-0077.2018.03.016

2018-01-01

Abstract:With the rapid growth of semantic data in recent years,the backward chaining reasoning method appears to be a new research direction since it is insensitive to data changes.Due to its complex reasoning procedure and large searching space of rule extensions,the backward chaining reasoning is still defected in the reasoning perform-ance and scalability.This paper makes a thorough analysis on the characteristics of semantic rules based upon previ-ous works about backward chaining reasoning technique,and proposes the design of a parallelized large-scale back-ward chaining reasoning engine of high efficiency and scalability over apache spark,the state-of-art big data process-ing platform. The main contributions of this paper can be summarized as:1)It avoids duplicate reasoning of terminological patterns during real-time reasoning by pre-calculating terminological closure;2)An optimization methods for reverse reasoning procedure and querying procedure is designed for an improved performance;3)Spark based implementa-tion of the proposed algorithm is presented.Experimental results on both synthetic datasets and real-world datasets show that our method requires only several seconds to ten-seconds of reasoning over hundreds of millions triples, maintaining high data scalability and node scalability as well.