Abstract:Background: Medical students must meet curricular expectations and pass national licensing examinations to become physicians. However, no previous studies explicitly modeled stages of medical students acquiring basic science knowledge. In this study, we employed an innovative statistical model to characterize students' growth using progress testing results over time and predict licensing examination performance. Methods: All students matriculated from 2016 to 2017 in our medical school with USMLE Step 1 test scores were included in this retrospective cohort study (N = 358). Markov chain method was employed to: 1) identify latent states of acquiring scientific knowledge based on progress tests and 2) estimate students' transition probabilities between states. The primary outcome of this study, United States Medical Licensing Examination (USMLE) Step 1 performance, were predicted based on students' estimated probabilities in each latent state identified by Markov chain model. Results: Four latent states were identified based on students' progress test results: Novice, Advanced Beginner I, Advanced Beginner II and Competent States. At the end of the first year, students predicted to remain in the Novice state had lower mean Step 1 scores compared to those in the Competent state (209, SD = 14.8 versus 255, SD = 10.8 respectively) and had more first attempt failures (11.5% versus 0%). On regression analysis, it is found that at the end of the first year, if there was 10% higher chance staying in Novice State, Step 1 scores will be predicted 2.0 points lower (95% CI: 0.85-2.81 with P < .01); while 10% higher chance in Competent State, Step 1scores will be predicted 4.3 points higher (95% CI: 2.92-5.19 with P < .01). Similar findings were also found at the end of second year medical school. Conclusions: Using the Markov chain model to analyze longitudinal progress test performance offers a flexible and effective estimation method to identify students' transitions across latent stages for acquiring scientific knowledge. The results can help identify students who are at-risk for licensing examination failure and may benefit from targeted academic support.

When can we identify the students at risk of failure in the national medical licensure examination in Japan using the predictive pass rate?

When can we identify the students at risk of failure in the national medical licensure examination in Japan using the predictive pass rate?

Predictors and students’ perceptions of NCLEX-RN success in a BS program

Predictors of First-Time NCLEX-RN Pass Rate in Accelerated Bachelor of Science in Nursing Students

Using Markov chain model to evaluate medical students' trajectory on progress tests and predict USMLE step 1 scores---a retrospective cohort study in one medical school

Early prediction of medical students' performance in high-stakes examinations using machine learning approaches

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan

Validation of the General Medicine in-Training Examination Using the Professional and Linguistic Assessments Board Examination Among Postgraduate Residents in Japan

The Analysis of the Backgrounds and Scores of Examinees Entering for Clinical Licensing Examination Multi-Times

The Application of Adaptive Minimum Match k-Nearest Neighbors to Identify At-Risk Students in Health Professions Education

Utilisation of Learning Analytics to Identify Students at Risk of Poor Academic Performance in Medical Schools

High Total Failures of the North American Pharmacist Licensure Examination (NAPLEX) Warrant Monitoring of Professional Pharmacy Programs

The Relationship of a Predictable Ability Measurement Examination and NCLEX-RN Disposition

Does Pass/Fail on Medical Licensing Exams Predict Future Physician Performance in Practice? A Longitudinal Cohort Study of Alberta Physicians

An Artificial Neural Network Based Early Prediction of Failure-Prone Students in Blended Learning Course

How Learning Strategies and Academic Parameters Predict Medical Student Success.

Potential Impact of Pass/Fail Scores on USMLE Step 1: Predictors of Excellence in Obstetrics and Gynecology Residency Training

Early identification of Family Medicine residents at risk of failure using Natural Language Processing and Explainable Artificial Intelligence

Relationship Between Standardized Test Scores and Board Certification Exams in a Combined Internal Medicine/Pediatrics Residency Program

Apply Machine Learning Algorithms to Predict At-Risk Students to Admission Period

Pass/Fail Prediction in Programming Courses