StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children's Story-Based Learning

Jiaju Chen,Yuxuan Lu,Shao Zhang,Bingsheng Yao,Yuanzhe Dong,Ying Xu,Yunyao Li,Qianwen Wang,Dakuo Wang,Yuling Sun

2024-10-04

Abstract:Interactive story reading is a common parent-child activity, where parents expect to teach both language skills and real-world knowledge beyond the story. While increasing storytelling and reading systems have been developed for this activity, they often fail to infuse real-world knowledge into the conversation. This limitation can be attributed to the existing question-answering (QA) datasets used for children's education, upon which the systems are built, failing to capture the nuances of how education experts think when conducting interactive story reading activities. To bridge this gap, we design an annotation framework, empowered by existing knowledge graph to capture experts' annotations and thinking process, and leverage this framework to construct StorySparkQA dataset, which comprises 5,868 expert-annotated QA pairs with real-world knowledge. We conduct automated and human expert evaluations across various QA pair generation settings to demonstrate that our StorySparkQA can effectively support models in generating QA pairs that target real-world knowledge beyond story content. StorySparkQA is available at <a class="link-external link-https" href="https://huggingface.co/datasets/NEU-HAI/StorySparkQA" rel="external noopener nofollow">this https URL</a>.

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively integrate real - world knowledge in children's story - reading interactions to support teachers or parents in guiding children to learn knowledge beyond the story content. Existing story - reading systems are often unable to integrate real - world knowledge well into conversations, mainly because the story - question - answering datasets for children's education fail to capture the thinking modes and nuances of educational experts during interactive story - reading activities. Therefore, this research aims to collect the knowledge and thinking processes of educational experts by designing an annotation framework supported by existing knowledge graphs, and use this framework to construct a dataset - StorySparkQA - containing 5,868 expert - annotated question - answer pairs, which include real - world knowledge. The research also shows through automatic evaluation and human - expert evaluation that StorySparkQA can effectively support models to generate question - answer pairs for real - world knowledge, thus making up for the deficiencies of existing systems and better meeting the expectations of teachers and parents.