Abstract:Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

Large Language Model (LLM) Bias Index -- LLMBI

Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs

Bias Neutralization Framework: Measuring Fairness in Large Language Models with Bias Intelligence Quotient (BiQ)

Measuring Implicit Bias in Explicitly Unbiased Large Language Models

LangBiTe: A Platform for Testing Bias in Large Language Models

MBIAS: Mitigating Bias in Large Language Models While Retaining Context

Bias Similarity Across Large Language Models

LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education

Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approach

Bias and Fairness in Large Language Models: A Survey

The life cycle of large language models in education: A framework for understanding sources of bias

Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective

Interpreting Bias in Large Language Models: A Feature-Based Approach

Revealing Hidden Bias in AI: Lessons from Large Language Models

Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models

Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models

Promoting Equality in Large Language Models: Identifying and Mitigating the Implicit Bias based on Bayesian Theory

A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions

Cognitive Bias in Decision-Making with LLMs

Exploring Subjectivity for more Human-Centric Assessment of Social Biases in Large Language Models