Abstract:When a teacher provides examples for a student to study, these examples must be informative, enabling a student to progress from their current state toward a target concept or skill. Good teachers must therefore simultaneously infer what students already know and adapt their teaching to students' changing state of knowledge. There is increasing interest in using computational models, particularly large language models, as pedagogical tools. As students, language models in particular have shown a remarkable ability to adapt to new tasks given small numbers of examples. But how effectively can these models adapt as teachers to students of different types? To study this question, we introduce a suite of models and evaluation methods we call AdapT. AdapT has two components: (1) a collection of simulated Bayesian student models that can be used for evaluation of automated teaching methods; (2) a platform for evaluation with human students, to characterize the real-world effectiveness of these methods. We additionally introduce (3) AToM, a new probabilistic model for adaptive teaching that jointly infers students' past beliefs and optimizes for the correctness of future beliefs. In evaluations of simulated students across three learning domains (fraction arithmetic, English morphology, function learning), AToM systematically outperforms LLM-based and standard Bayesian teaching models. In human experiments, both AToM and LLMs outperform non-adaptive random example selection. Our results highlight both the difficulty of the adaptive teaching task and the potential of learned adaptive models for solving it.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to make teaching adapt to students' specific misunderstandings and prior knowledge, especially when using large language models (LLMs) as teaching tools. Specifically, the paper explores how to utilize computational models, especially large language models, to achieve "in - context teaching", that is, to select the most appropriate teaching examples according to students' current knowledge state and misunderstandings, so as to help students learn the target concepts or skills more effectively. ### Background and Objectives of the Paper The paper points out that a good teacher needs to infer what students already know at the same time and adjust teaching methods according to students' constantly changing knowledge states. Although existing large language models can quickly adapt to new tasks with a small number of examples when acting as students, whether they can effectively adapt to different types of students when acting as teachers is still a problem to be studied. ### Main Contributions To study this problem, the author introduced an evaluation framework named ADAPT, which consists of two parts: 1. **Simulated Student Model**: It is used to evaluate the effectiveness of automated teaching methods. 2. **Human Student Evaluation Platform**: It is used to evaluate the effectiveness of these methods in the real world. In addition, the author also proposed a new probability model - ATOM (Adaptive Teaching tOwards Misconceptions), which realizes adaptive teaching by jointly inferring students' past beliefs and optimizing the correctness of future beliefs. ### Experimental Results In the evaluation of simulated students, ATOM systematically outperforms methods based on LLM and standard Bayesian teaching models in three learning areas: fractional arithmetic, English morphology, and function learning. In human experiments, both ATOM and LLM are superior to the non - adaptive random example selection method, which indicates that adaptive teaching methods have great potential. ### Conclusions The paper emphasizes the difficulty of the adaptive teaching task and the potential of using the learned adaptive model to solve this problem. Through the ADAPT framework and the ATOM model, the author shows how to use computational models to more effectively identify and correct students' misunderstandings, thereby improving teaching effectiveness.

Toward In-Context Teaching: Adapting Examples to Students' Misconceptions

Utility-based Adaptive Teaching Strategies using Bayesian Theory of Mind

Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners

Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization

Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

Toward a general, scaleable framework for Bayesian teaching with applications to topic models

Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes

Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories

LLM-based Cognitive Models of Students with Misconceptions

Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning

Iterative Teacher-Aware Learning

How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging

Adaptive teaching in mathematics: a review of the literature

Evaluating and Optimizing Educational Content with Large Language Model Judgments

Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study

Adaptive Teaching of Temporal Logic Formulas to Learners with Preferences

Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers

AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails

Earning to t each

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions