Abstract:BackgroundHigh-stakes assessments, such the Graduate Records Examination, have transitioned from paper to computer administration. Low-stakes research-based assessments (RBAs), such as the Force Concept Inventory, have only recently begun this transition to computer administration with online services. These online services can simplify administering, scoring, and interpreting assessments, thereby reducing barriers to instructors’ use of RBAs. By supporting instructors’ objective assessment of the efficacy of their courses, these services can stimulate instructors to transform their courses to improve student outcomes. We investigate the extent to which RBAs administered outside of class with the online Learning About STEM Student Outcomes (LASSO) platform provide equivalent data to tests administered on paper in class, in terms of both student participation and performance. We use an experimental design to investigate the differences between these two assessment conditions with 1310 students in 25 sections of 3 college physics courses spanning 2 semesters.ResultsAnalysis conducted using hierarchical linear models indicates that student performance on low-stakes RBAs is equivalent for online (out-of-class) and paper-and-pencil (in-class) administrations. The models also show differences in participation rates across assessment conditions and student grades, but that instructors can achieve participation rates with online assessments equivalent to paper assessments by offering students credit for participating and by providing multiple reminders to complete the assessment.ConclusionsWe conclude that online out-of-class administration of RBAs can save class and instructor time while providing participation rates and performance results equivalent to in-class paper-and-pencil tests.

Comparing Test-Taking Effort Between Paper-Based and Computer-Based Tests

Does Medical Students' Preference of Test Format (Computer-based vs. Paper-based) have an Influence on Performance?

Comparing computer-based and paper-based rating modes in an English writing test

Examining mode effects for an adapted Chinese critical thinking assessment

Impact of Computer-Based Assessments on the Science's Ranks of Secondary Students

Moving a formative test from a paper-based to a computer-based format. A student viewpoint

From paper to pixels: A comparison of paper and computer formats in psychological assessment

Does Timed Testing Affect the Interpretation of Efficiency Scores?—A GLMM Analysis of Reading Components

Computerized testing in reading comprehension skill: investigating score interchangeability, item review, age and gender stereotypes, ICT literacy and computer attitudes

Computer-based and paper-and-pencil tests: A study in calculus for STEM majors

Comparative Study on Validity of Paper-Based Test and Computer-Based Test in the Context of Educational and Psychological Assessment among Arab Students

Participation and performance on paper- and computer-based low-stakes assessments

Exploring Motivational Factors Associated With Test-Taking Effort Among Middle School Students

Measurement precision and user experience with adaptive versus non-adaptive psychometric tests

Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity

Validity of Computer Based Administration of Cognitive Assessments compared to Traditional Paper-based Administration

Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions.

A Comparison of Two Methods of Administering Group Reading Inventories to Diverse Learners

A study of validity and reliability of Internet-based English writing testing

Assessing the Speed–Accuracy Tradeoff in Psychological Testing Using Experimental Manipulations

Assessing time-based prospective memory online: A comparison study between laboratory-based and web-based testing