SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases
Sumit Neelam,Udit Sharma,Hima Karanam,Shajith Ikbal,Pavan Kapanipathi,Ibrahim Abdelaziz,Nandana Mihindukulasooriya,Young-Suk Lee,Santosh Srivastava,Cezar Pendus,Saswati Dana,Dinesh Garg,Achille Fokoue,G P Shrivatsa Bhargav,Dinesh Khandelwal,Srinivas Ravishankar,Sairam Gurajada,Maria Chang,Rosario Uceda-Sosa,Salim Roukos,Alexander Gray,Guilherme LimaRyan Riegel,Francois Luus,L Venkata Subramaniam
DOI: https://doi.org/10.48550/arXiv.2109.13430
2021-09-28
Abstract:Knowledge Base Question Answering (KBQA) tasks that in-volve complex reasoning are emerging as an important re-search direction. However, most KBQA systems struggle withgeneralizability, particularly on two dimensions: (a) acrossmultiple reasoning types where both datasets and systems haveprimarily focused on multi-hop reasoning, and (b) across mul-tiple knowledge bases, where KBQA approaches are specif-ically tuned to a single knowledge base. In this paper, wepresent SYGMA, a modular approach facilitating general-izability across multiple knowledge bases and multiple rea-soning types. Specifically, SYGMA contains three high levelmodules: 1) KB-agnostic question understanding module thatis common across KBs 2) Rules to support additional reason-ing types and 3) KB-specific question mapping and answeringmodule to address the KB-specific aspects of the answer ex-traction. We demonstrate effectiveness of our system by evalu-ating on datasets belonging to two distinct knowledge bases,DBpedia and Wikidata. In addition, to demonstrate extensi-bility to additional reasoning types we evaluate on multi-hopreasoning datasets and a new Temporal KBQA benchmarkdataset on Wikidata, namedTempQA-WD1, introduced in thispaper. We show that our generalizable approach has bettercompetetive performance on multiple datasets on DBpediaand Wikidata that requires both multi-hop and temporal rea-soning
Computation and Language,Artificial Intelligence