Abstract:This paper presents an out-of-sample prediction comparison between major machine learning models and the structural econometric model. Over the past decade, machine learning has established itself as a powerful tool in many prediction applications, but this approach is still not widely adopted in empirical economic studies. To evaluate the benefits of this approach, I use the most common machine learning algorithms, CART, C4.5, LASSO, random forest, and adaboost, to construct prediction models for a cash transfer experiment conducted by the Progresa program in Mexico, and I compare the prediction results with those of a previous structural econometric study. Two prediction tasks are performed in this paper: the out-of-sample forecast and the long-term within-sample simulation. For the out-of-sample forecast, both the mean absolute error and the root mean square error of the school attendance rates found by all machine learning models are smaller than those found by the structural model. Random forest and adaboost have the highest accuracy for the individual outcomes of all subgroups. For the long-term within-sample simulation, the structural model has better performance than do all of the machine learning models. The poor within-sample fitness of the machine learning model results from the inaccuracy of the income and pregnancy prediction models. The result shows that the machine learning model performs better than does the structural model when there are many data to learn; however, when the data are limited, the structural model offers a more sensible prediction. The findings of this paper show promise for adopting machine learning in economic policy analyses in the era of big data.

Assessing the effectiveness of special education services on fifth grade math scores: Using traditional and machine learning methods with ECLS-K data

Assessing the Causal Effect of Special Education Services on Math Achievement: A Causal Inference and Machine Learning Study

Machine learning and propensity score matching for evaluating the effect of special education services on childrens later math performances

The Impact of Online Computer Assisted Learning at Home for Disadvantaged Children in Taiwan: Evidence from a Randomized Experiment

Estimating returns to special education: combining machine learning and text analysis to address confounding

The impact of a technology-based mathematics after-school program using ALEKS on student's knowledge and behaviors

Special Education Participation in Statewide Accountability Assessment: Analysis of Options Using Item Response Theory.

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Educational Technology in Support of Elementary Students With Reading or Language-Based Disabilities: A Cluster Randomized Control Trial

USING A MACHINE LEARNING APPROACH TO EXPLORE NON-COGNITIVE FACTORS AFFECTING READING, MATHEMATICS, AND SCIENCE LITERACY IN CHINA AND THE UNITED STATES

Does Computer-Assisted Learning Improve Learning Outcomes? Evidence from a Randomized Experiment in Migrant Schools in Beijing

Do Written Responses to Open-Ended Questions on Fourth-Grade Online Formative Assessments in Mathematics Help Predict Scores on End-of-Year Standardized Tests?

Do Mathematics and Reading Skills Impact Student Science Outcomes?

Long-Term Effects of Social–Emotional Learning on Receipt of Special Education and Grade Retention: Evidence From a Randomized Trial of INSIGHTS

Integrating computer-assisted learning into a regular curriculum: evidence from a randomised experiment in rural schools in Shaanxi

Growth in Mathematics Achievement: Analysis With Classification and Regression Trees

Computer assisted learning as extracurricular tutor? Evidence from a randomised experiment in rural boarding schools in Shaanxi

Improving predictive power through deep learning analysis of K-12 online student behaviors and discussion board content

Evaluating Conditional Cash Transfer Policies with Machine Learning Methods

Classroom quality and academic skills: Approaches to learning as a moderator.

More is Not Always Better: Evidence from a Randomised Experiment of Computer-Assisted Learning in Rural Minority Schools in Qinghai