Practice Makes Better: Quantifying Grade Benefits of Study
William K. Black,Rebecca L. Matz,Mark Mills,A. E. Evrard
DOI: https://doi.org/10.48550/arXiv.2301.02927
2023-01-07
Physics Education
Abstract:Problem Roulette (PR), an online study service at the University of Michigan, offers point-free practice to students preparing for examinations in introductory STEM courses. Using four years of PR data involving millions of problem attempts by thousands of students, we quantify benefits of increased practice study volume in introductory physics. After conditioning mean final grade on standardized (ACT/SAT) math test score, we analyze deviations based on student study volume. We find a strong effect; mean course grade rises quadratically with the logarithm of the total number of PR questions encountered over the term ($N_{\rm Q, tot}$), with an overall gain of $0.78 \pm 0.12$ grade points between $1 < N_{\rm Q, tot} < 1000$. The gains are persistent across the range of math test score represented in our sample. A linear model for final grade using test score and study volume largely accounts for demographic stratification, including by sex, parental education level, number of parents at home, nationality / underrepresented minority status, and regional income level, with two significant exceptions: students whose parents did not earn a college degree, who earn $-0.27 \pm 0.04$ grade points ($6.2\sigma$) below expectations and underrepresented minority students, at $-0.14 \pm 0.04$ points ($3.7\sigma$). Residual scatter in final grade remains comparable to the maximum gain from studying more, implying that the model is far from deterministic at the level of an individual. Our findings can help motivate students to study more and help teachers to identify which types of students may especially need such encouragement.