Abstract:A hallmark property of explainable AI models is the ability to teach other agents, communicating knowledge of how to perform a task. While Large Language Models perform complex reasoning by generating explanations for their predictions, it is unclear whether they also make good teachers for weaker agents. To address this, we consider a student-teacher framework between two LLM agents and study if, when, and how the teacher should intervene with natural language explanations to improve the student's performance. Since communication is expensive, we define a budget such that the teacher only communicates explanations for a fraction of the data, after which the student should perform well on its own. We decompose the teaching problem along four axes: (1) if teacher's test time intervention improve student predictions, (2) when it is worth explaining a data point, (3) how the teacher should personalize explanations to better teach the student, and (4) if teacher explanations also improve students on future unexplained data. We first show that teacher LLMs can indeed intervene on student reasoning to improve their performance. Next, inspired by the Theory of Mind abilities of effective teachers, we propose building two few-shot mental models of the student. The first model defines an Intervention Function that simulates the utility of an intervention, allowing the teacher to intervene when this utility is the highest and improving student performance at lower budgets. The second model enables the teacher to personalize explanations for a particular student and outperform unpersonalized teachers. We also demonstrate that in multi-turn interactions, teacher explanations generalize and learning from explained data improves student performance on future unexplained data. Finally, we verify that misaligned teachers can lower student performance to random chance by intentionally misleading them.

Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning

Pedagogical Alignment of Large Language Models

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions

Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors

LLM-based Cognitive Models of Students with Misconceptions

Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization

LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education

The Life Cycle of Large Language Models: A Review of Biases in Education

Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models

"The teachers are confused as well": A Multiple-Stakeholder Ethics Discussion on Large Language Models in Computing Education

Large Language Models for In-Context Student Modeling: Synthesizing Student's Behavior in Visual Programming

Leveraging Prompts in LLMs to Overcome Imbalances in Complex Educational Text Data

LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations

Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

Language Models as Science Tutors

Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study

Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course