Abstract:Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

Ethical Considerations for Companies Implementing LLMs in Education Software

Navigating LLM Ethics: Advancements, Challenges, and Future Directions

"The teachers are confused as well": A Multiple-Stakeholder Ethics Discussion on Large Language Models in Computing Education

Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics

Ethical Considerations and Fundamental Principles of Large Language Models in Medical Education: Viewpoint

Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models

The Ethics of Interaction: Mitigating Security Threats in LLMs

Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

The life cycle of large language models in education: A framework for understanding sources of bias

Large language models (LLMs): Risks and policy implications

The Only Way is Ethics: A Guide to Ethical Research with Large Language Models

The Life Cycle of Large Language Models: A Review of Biases in Education

Ethical and regulatory challenges of large language models in medicine

A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law

The global landscape of academic guidelines for generative AI and Large Language Models

"I Always Felt that Something Was Wrong.": Understanding Compliance Risks and Mitigation Strategies when Professionals Use Large Language Models

Clinician voices on ethics of LLM integration in healthcare: a thematic analysis of ethical concerns and implications

"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices

Adapting to the Impact of AI in Scientific Writing: Balancing Benefits and Drawbacks while Developing Policies and Regulations

LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education