Abstract:In educational assessment, Multiple Choice Questions (MCQs) are frequently used due to their efficiency in grading and providing feedback. However, manual MCQ generation encounters challenges. Relying on a limited set of questions may lead to item repetition, which could compromise the reliability of assessments and the security of the evaluation procedure, especially in high-stakes evaluations. This study explores an AI-driven approach to creating and evaluating MCQs in introductory chemistry and biology. The methodology involves generating Bloom’s Taxonomy-aligned questions through zero-shot prompting with GPT-3.5, validating question alignment with Bloom’s Taxonomy with RoBERTa–a language model grounded in transformer architecture, employing self-attention mechanisms to handle input sequences and produce context-aware representations of individual words within a given sentence–, evaluating question quality using Item Writing Flaws (IWF)–issues that can arise in the creation of test items or questions–, and validating questions using subject matter experts. Our research demonstrates GPT-3.5’s capacity to produce higher-order thinking questions, particularly at the "evaluation" level. We observe alignment between GPT-generated questions and human-assessed complexity, albeit with occasional disparities. Question quality assessment reveals differences between human and machine evaluations, correlating inversely with Bloom’s Taxonomy levels. These findings shed light on automated question generation and assessment, presenting the potential for advancements in AI-driven educational evaluation methods.

Comprehension Based Question Answering using Bloom's Taxonomy

BloomVQA: Assessing Hierarchical Multi-modal Comprehension

Towards AI-Assisted Multiple Choice Question Generation and Quality Evaluation at Scale: Aligning with Bloom’s Taxonomy

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Contextualized Representations Using Textual Encyclopedic Knowledge

Evaluating English Textbooks by Using Bloom’s Taxonomy to Analyze Reading Comprehension Question

CompPrehension - Model-Based Intelligent Tutoring System on Comprehension Level

How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes

Careful Selection of Knowledge to solve Open Book Question Answering

Adaptable Closed-Domain Question Answering Using Contextualized CNN-Attention Models and Question Expansion

Auxiliary Task Guided Interactive Attention Model for Question Difficulty Prediction

How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering

BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts

Open Domain Question Answering Via Semantic Enrichment

Evaluating Commonsense in Pre-trained Language Models

BB-KBQA: BERT-Based Knowledge Base Question Answering

Generating Question-Answer Hierarchies

Keep Skills in Mind: Understanding and Implementing Skills in Commonsense Question Answering

Complex Knowledge Base Question Answering: A Survey

OpenCQA: Open-ended Question Answering with Charts

Knowledge Base Question Answering: A Semantic Parsing Perspective