Automating question generation from educational text

Ayan Kumar Bhowmick,Ashish Jagmohan,Aditya Vempaty,Prasenjit Dey,Leigh Hall,Jeremy Hartman,Ravi Kokku,Hema Maheshwari

DOI: https://doi.org/10.48550/arXiv.2309.15004

2023-09-26

Abstract:The use of question-based activities (QBAs) is wide-spread in education, traditionally forming an integral part of the learning and assessment process. In this paper, we design and evaluate an automated question generation tool for formative and summative assessment in schools. We present an expert survey of one hundred and four teachers, demonstrating the need for automated generation of QBAs, as a tool that can significantly reduce the workload of teachers and facilitate personalized learning experiences. Leveraging the recent advancements in generative AI, we then present a modular framework employing transformer based language models for automatic generation of multiple-choice questions (MCQs) from textual content. The presented solution, with distinct modules for question generation, correct answer prediction, and distractor formulation, enables us to evaluate different language models and generation techniques. Finally, we perform an extensive quantitative and qualitative evaluation, demonstrating trade-offs in the use of different techniques and models.

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automatically generate text - based multiple - choice questions (MCQs) in education, so as to reduce the workload of teachers and promote personalized learning experiences. Specifically:

1. **Teacher Workload**: Traditional question - based activities (QBAs) are widely used in education, but preparing these activities is time - consuming and labor - intensive. By surveying 104 teachers, the paper reveals the challenges that teachers face when preparing QBAs, such as lack of time, lack of high - quality resources, and lack of personalized adaptive resources.

2. **Automated Generation Technology**: Utilize recent generative artificial intelligence (AI) technologies, especially Transformer - based language models (such as T5 and GPT - 3), to develop an automated MCQ generation tool. This tool can automatically generate multiple - choice questions from educational texts, including three main modules: question generation, correct - answer prediction, and distractor generation.

3. **System Evaluation**: Conduct quantitative and qualitative evaluations of the generated MCQs to verify the effectiveness of the system and the quality of the generated questions. Evaluation metrics include perplexity, query well - formedness, answer accuracy (exact match, F1 - score, ROUGE), and distractor quality.

4. **Modular Framework**: Propose a modular framework that allows independent development and optimization of each module and integrates different language models to meet various educational needs. This flexibility not only improves the overall quality of MCQs but also makes the system more robust, scalable, and highly customizable.

In conclusion, this paper aims to help teachers reduce their workload while improving teaching quality and students' learning experiences by developing an efficient, reliable, and privacy - protected automatic MCQ generation system.