Crebot: Exploring Interactive Question Prompts for Critical Paper Reading

Zhenhui Peng,Yuzhi Liu,Hanqi Zhou,Zuyu Xu,Xiaojuan Ma

DOI: https://doi.org/10.2139/ssrn.4001460

2022-01-01

SSRN Electronic Journal

Abstract:Pre-compiled guidelines with a static question list can stimulate critical thinking during scientific paper reading. However, they could be less engaging than taking live question prompts from others. In this paper, we develop CReBot that interactively asks section-level critical thinking questions and customize it for routine paper readers with prior research experience and novices new to research. Our first within-subjects study with 24 routine readers demonstrates CReBot's engagement and usefulness over static guidelines. Then, with more teacher-like question-specific hints prepared for CReBot, we conduct another within-subjects study with 20 novices. The results, however, indicate that CReBot might not be better than static guidelines for beginners. Nevertheless, both user groups favor CReBot's contextualized questions and interaction flexibility. We conclude by design implications for interactive tools to facilitate critical reading.