Cross-Level Mediation in School-Randomized Studies of Teacher Development: Experimental Design and Power

Ben Kelcey,Jessaca Spybrook,Nianbo Dong,Fangxing Bai
DOI: https://doi.org/10.1080/19345747.2020.1726540
2020-05-28
Journal of Research on Educational Effectiveness
Abstract:Professional development for teachers is regarded as one of the principal pathways through which we can understand and cultivate effective teaching and improve student outcomes. A critical component of studies that seek to improve teaching through professional development is the detailed assessment of the intermediate teacher development processes that scaffold program content through three key types of outcomes—teacher knowledge, instruction, and student learning. Cross-level and sequential mediation strategies that probe and connect these processes and outcomes emerge as an important design consideration in these studies. We derive formulas that track the power with which school-randomized designs can detect professional development effects as they operate through a sequence of teacher-level mediators to affect student outcomes (e.g., school-randomized professional development studies). The results suggest that the sample sizes typically seen in well-planned experiments targeting total effects (e.g., 30–100 schools) can produce comparably high or disparately low levels of power for mediation effects—the similarity depends heavily on context and concomitant parameter values. The results are implemented in the PowerUpR package and in the PowerUpR Shiny application.
education & educational research
What problem does this paper attempt to address?