Abstract:BackgroundEconomists are making increasing use of measures of student achievement obtained through large-scale survey assessments such as NAEP, TIMSS, and PISA. The construction of these measures, employing plausible value (PV) methodology, is quite different from that of the more familiar test scores associated with assessments such as the SAT or ACT. These differences have important implications both for utilization and interpretation. Although much has been written about PVs, it appears that there are still misconceptions about whether and how to employ them in secondary analyses.MethodsWe address a range of technical issues, including those raised in a recent article that was written to inform economists using these databases. First, an extensive review of the relevant literature was conducted, with particular attention to key publications that describe the derivation and psychometric characteristics of such achievement measures. Second, a simulation study was carried out to compare the statistical properties of estimates based on the use of PVs with those based on other, commonly used methods.ResultsIt is shown, through both theoretical analysis and simulation, that under fairly general conditions appropriate use of PV yields approximately unbiased estimates of model parameters in regression analyses of large scale survey data. The superiority of the PV methodology is particularly evident when measures of student achievement are employed as explanatory variables.ConclusionsThe PV methodology used to report student test performance in large scale surveys remains the state-of-the-art for secondary analyses of these databases.

Aggregating Student Achievement Trends Across States With Different Tests: Using Standardized Slopes as Effect Sizes

Making the grade: The effect of teacher grading standards on student outcomes

Longitudinal Effect of the Washington Assessment of Student Learning (WASL) on Student Achievement

The use of test scores from large-scale assessment surveys: psychometric and statistical considerations

Measurement Invariance and Cohort Trends for Social and Emotional Learning Measures Across Four Statewide Administrations: Conventional Fit Statistics Versus the RMSEAD

Discussion of "on State Education Statistics": A Difficulty with Regression Analyses of Regional Test Score Averages

Visualizing Large-Scale Data in Educational, Behavioral, Psychometrical and Social Sciences: Utilities and Design Patterns of the SEER Computational and Graphical Statistics

Assessing the Comparability of School Scores Across Test Forms That Are Not Parallel

High-Achieving and Average Students’ Reading Growth: Contrasting School and Summer Trajectories

A quantitative review of competing learning slope metrics: effects of age, sex, and clinical diagnosis

Is Student Achievement Immutable? Evidence From International Studies on Schooling and Student Achievement

Toward Measuring Instructional Interactions “At-Scale”

Extracting candidate factors affecting long-term trends of student abilities across subjects

New Approaches for Testing Slope Homogeneity in Large Panel Data Models

The Analysis Of Examination Scores

Standard Errors for National Trends in International Large-Scale Assessments in the Case of Cross-National Differential Item Functioning

Patterns in Illinois Educational School Data

Linking Large-Scale Reading Assessments: Measuring International Trends Over 40 Years

Approaches to Statistical Efficiency When Comparing the Embedded Adaptive Interventions in a SMART

The Causal Effect of Student Mobility on Standardized Test Performance: A Case Study with Possible Implications for Accountability Mandates within the Elementary and Secondary Education Act

The influence of procedural characteristics on within-case effect sizes for academic outcomes