Abstract:National educational standards stress the importance of science and mathematics learning for today's students. However, across disciplines, students frequently struggle to meet learning goals about core concepts like energy. Digital learning environments enhanced with artificial intelligence hold the promise to address this issue by providing individualized instruction and support for students at scale. Scaffolding and feedback, for example, are both most effective when tailored to students' needs. Providing individualized instruction requires continuous assessment of students' individual knowledge, abilities, and skills in a way that is meaningful for providing tailored support and planning further instruction. While continuously assessing individual students' science and mathematics learning is challenging, intelligent tutoring systems show that it is feasible in principle. However, the learning environments in intelligent tutoring systems are typically not compatible with the vision of how effective K-12 science and mathematics learning looks like. This leads to the challenge of designing digital learning environments that allow for both – meaningful science and mathematics learning and the reliable and valid assessment of individual students' learning. Today, digital devices such as tablets, laptops, or digital measurement systems increasingly enter science and mathematics classrooms. In consequence, students' learning increasingly produces rich product and process data. Learning Analytics techniques can help to automatically analyze this data in order to obtain insights about individual students' learning, drawing on general theories of learning and relative to established domain specific models of learning, i.e., learning progressions. We call this approach Learning Progression Analytics (LPA). In this manuscript, building of evidence-centered design (ECD), we develop a framework to guide the development of learning environments that provide meaningful learning activities and data for the automated analysis of individual students' learning – the basis for LPA and scaling individualized instruction with artificial intelligence.

Use of Evidence-Centered Design to Develop Learning Maps-Based Assessments

Utilizing Evidence-Centered Design to Develop Assessments: A High School Introductory Computer Science Course

Developing Assessments of Content Knowledge for Teaching Using Evidence-centered Design

The Expanded Evidence-Centered Design (e-ECD) for Learning and Assessment Systems: A Framework for Incorporating Learning Goals and Processes Within Assessment Design

Representing Targets of Measurement Within Evidence-Centered Design

Leveraging Evidence-Centered Design to Develop Assessments of Computational Thinking Practices

Concept maps as assessment for learning in university mathematics

A design template for transforming games into twenty-first century skills assessments

Developing a kindergarten computational thinking assessment using evidence-centered design: the case of algorithmic thinking

Utilizing Cognitive Load Theory and Evidence-Centered Design to Inform the Design of Game-Based Learning Environments

Validity evidence for programmatic assessment in competency-based education

Toward learning progression analytics — Developing learning environments for the automated analysis of learning using evidence centered design

On the Development of a Computer-Based Tool for Formative Student Assessment: Epistemological, Methodological, and Practical Issues

Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research

Game-Based Assessment of Students' Digital Literacy Using Evidence-Centered Game Design

Getting at Student Understanding - the Key to Teachers’ Use of Test Data

Challenges for the Design of International Assessments: Sampling, Measurement, and Causality

A Game-Based Online Tool to Measure Cognitive Functions in Students

ECBD: Evidence-Centered Benchmark Design for NLP

Assessing Scientific Practices in Physics Paper-based Assessments

What Type of Self-Assessment Is Best for Your Educational Activity? A Review of Pre-Post, Now-Then, and Post-Only Designs