Quizlet and the Testing Effect 1 The Self-Study Testing Effect with Bidirectional Paired Associate Learning
B. Barton
Abstract:The testing effect describes a method of retrieval practice that benefits long term retention of learned material, over restudying and re-exposure (Roediger & Karpicke, 2006a). The benefits of testing have been implemented outside of psychology laboratories into classrooms around the US, but are still not being taken advantage of by students outside of lecture (Karpicke, Butler & Roediger, 2009). The current experiment seeks to acknowledge the testing effect as present in a widely used self-study website and further define the most effective retrieval practice students can implement within it. Fifty-eight introductory Psychology students at the University of Colorado, Boulder studied 30 Anthropology terms and their definitions in a paired associate learning format on Quizlet.com. The paired associates were split into different self-study modes on Quizlet.com: one restudy block, and two cued recall with feedback blocks that applied either a consistent direction of study to initial presentation, or inconsistent direction. Two days after the self-study session, participants took a final retention test to measure the retention of the learned paired associates. The test analyzed the bidirectional retention of the 30-paired associates with recognition questions and cued-recall questions that manipulated direction; majority of questions included the paired associates verbatim to the study session, some of the questions were not verbatim to initial study but rather applied the paired associate relationships to an example to measure for comprehensive, conceptual understanding. The results of this study were inconclusive due to a confound in study block and question items, but paved way for future implications for studies on real self-study methods of students today, and the importance of retrieval practice direction on the direction of final retention. Quizlet and the Testing Effect 3 Introduction In educational settings, tests are administered by teachers to evaluate and measure their students retention of learned information. In psychological research, decades have been dedicated to studying the benefits of self testing as a retrieval practice to enhance final retention; this has been coined as: the testing effect (Roediger & Karpicke, 2006a). The extensively studied phenomenon depicts an advantage of self testing over restudying for long term retention and heightened probability of recall, confirming self testing benefits go beyond advantages of reexposure (Carrier & Pashler, 1992; Roediger & Karpicke, 2006a; Storm, Friedman, Murayama, & Bjork, 2014). The testing effect has been studied across various disciplines of Psychology and Neuroscience, all of which have concluded it occurs because of an active retrieval practice that modifies and enhances the memory of the retrieved information (Young, 2015). By acting as a consolidation event, the tested information is more likely to be recalled in the future than if the information was not used in retrieval practice (Storm, Friedman, Murayama, & Bjork, 2014). Although the testing effect is widely agreed upon and has yielded consistent significant effects, those who should benefit most from the implications are not taking advantage of it. A 2009 survey of 177 students at Washington University in St. Louis found that more than half of the students (55%) rated “rereading notes or textbooks” as the most important way to study for an exam, while only 1% rated self-testing as their most important study strategy (Karpicke, Butler & Roediger, 2009). A recent study considered that students are not implementing retrieval practice into their typical exam study due to lack of awareness of benefits. This study aimed to educate college students about the testing effect in hopes that this knowledge would self-direct them to self-test in the future (Young, 2015). The 2015 study split a research methods and statistics course into two groups: one group saw a PowerPoint presentation about the importance Quizlet and the Testing Effect 4 of self-testing and the other did not. Despite the lecture on benefits of self-testing, Young found no significant difference between groups for study methods or final grades. In two follow-up experiments, Young (2015) found that students who benefited from experimental test conditions did not implement the learned testing strategies on later study conditions. Clearly, the testing effect cannot be elicited by direction from teachers or subconscious practice, but it can be implemented in the classroom itself. Teachers are putting self-testing practice into action throughout the country with the use of iClickers. iClickers are an interactive technology used in over 700 college campuses, which allow for teachers to devote class time to testing material (Anderson, Healy, Kole & Bourne, 2011). Although these recent studies encourage teachers to devote class time to retrieval practice (Anderson et. al, 2011; Young, 2015), there are no studies that implement current technology available to students that allows them to practice retrieval outside of lecture. The testing effect has been studied extensively and proven within laboratory and classroom settings, but has yet to take advantage of the technology available to students outside the classroom. Even though students do not report using retrieval practice to study (Karpicke, Butler & Roediger, 2009), it is likely many students are already taking advantage of the benefits of self-testing without being aware of it. Today, there are multiple online platforms that allow students to input information from their courses and study in various formats on their personal computers. A meta-analysis of 159 testing effect articles found that the most pronounced testing effect occurs from a cued-recall retrieval practice, where retrieval of one item is enhanced by a cue to that item, that provides positive and negative feedback of the recalled item (Rowland, 2014). The current study uses modes that mirror this conclusion that already exist on a common Quizlet and the Testing Effect 5 online study site to observe pronounced testing effects in an environment more familiar to students and their current self-study habits. Paired associate formats in testing effect literature assign a stimulus to a specific response, and then cue recall of one with its corresponding pair. Of the 159 testing effect articles, over half of the studies with significant results employed a paired associate learning format for their studies (Rowland, 2014). The current study uses a common online study website: Quizlet. The website allows its 2 million monthly users to create digital “sets” (paired associates) of to be learned information, where one term corresponds to one definition. The website provides different study modes, one in a restudy format (“note card”), and another a cued recall test with positive and negative feedback (“learn”) to study the “sets” of paired associates. The paired associate testing effect studies in Rowland’s (2015) meta-analysis involve a unidirectional retrieval practice and do not manipulate the direction of retrieval practice or retention but rather only the type of study mode, test or restudy. The current study has the unique opportunity to determine if the testing effect exists in online self-study aids, and also to determine the degree to which direction of retrieval practice can improve the bidirectional retention and transfer of learned paired associate relationships. A short term memory study of bidirectional transfer of paired associates found that unidirectional forward retrieval practice produces a reverse retention of the pair only 70-90% as strong as the studied forward associate pair (Leuba, 1966). If there is not a 100% bidirectional transfer of associations in short term memory, it can be assumed that this hindered transfer will only decline with delay, alongside overall retention decline. Quizlet’s paired associate “sets” allow for bidirectional test study blocks. After the initial presentation of stimulus (term) to response (definition), the current experiment used Quizlet’s “learn” mode to manipulate two Quizlet and the Testing Effect 6 different study blocks: one in a consistent direction with the original presentation, and another inconsistent with the original direction. The retention test contained questions consistent and inconsistent with direction of study per block (see Appendix) to examine if the direction of retrieval practice influences the direction of retention and if the bidirectional transfer of one study direction is superior. In addition to forward and reverse direction questions, the retention test contained comprehension questions that were not verbatim to the paired associate study blocks and did not manipulate direction of test. Instead, these questions applied the term with an example to test for conceptual understanding (see Appendix). By the active retrieval practice of the testing study conditions, terms were likely affected by the generation effect, which says that generating stimuli increases the retention of that information (Rowland, 2014). The terms in the testing study blocks were expected to benefit from active generation of the term or the definition. A 2004 study of retrieval practice also manipulated subjects self-generation; in this study, subjects saw both complete and incomplete highlighted words in prose format and retained more of the incomplete words in a fill-in-the-blank retention test. This study showed a benefit of generation on retention because the active completion of incomplete highlighted words increased their probability of recall (deWinstanley and Bjork, 2004). As opposed to iClicker retrieval practice (usually recognition, multiple choice questions), the current online retrieval practice did not only provide a self-test method that benefits from an optimal retrieval practice, but from the advantage of self-generation of a response, prior to the positive and