Abstract:Despite its critical role for the development of the field, little is known about replication in second language (L2) research. To better understand replication practice, we first provide a narrative review of challenges related to replication, drawing on recent developments in psychology. This discussion frames and motivates a systematic review, building on syntheses of replication in psychology, education, and L2 research. We coded 67 self‐labeled L2 replication studies found across 26 journals for 136 characteristics. We estimated a mean rate of 1 published replication study for every 400 articles, with a mean of 6.64 years between initial and replication studies and a mean of 117 citations of the initial study before a replication was published. Replication studies had an annual mean of 7.3 citations, much higher than averages in linguistics and education. Overlap in authorship between initial and replication studies and the availability of the initial materials both increased the likelihood of a replication supporting the initial findings. Our sample contained no direct (exact) replication attempts, and changes made to initial studies were numerous and wide ranging, which likely obscured, if not undermined, the interpretability of replication studies. To improve the amount and quality of L2 replication research, we propose 16 recommendations relating to rationale, nomenclature, design, infrastructure, and incentivization for collaboration and publication. This article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the IRIS Repository at https://www.iris-database.org/iris/app/home/detail?id=york:934328. Learn more about the Open Practices badges from the Center for Open Science: https://osf.io/tvyxz/wiki.

Recommendations for Replication Research in Special Education

Replication Research and Special Education

Replication of Special Education Research

Replication Studies Using Secondary or Nonexperimental Datasets

Replication is important for educational psychology: Recent developments and key issues

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

Research into evidence-based psychological interventions needs a stronger focus on replicability

You Can't Publish Replication Studies (and How to Anyways)

Replication and Reproducibility in Cross-Cultural Psychology

Interdisciplinary lessons and recommendations for the evaluation of replicability in behavioral sciences

Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences

Large Scale Replication Projects in Contemporary Psychological Research

Replication studies in the Netherlands: Lessons learned and recommendations for funders, publishers and editors, and universities

Registered reports and replications: An ongoing Journal of School Psychology initiative

Prevalence of Research Design in Special Education: A Survey of Peer-Reviewed Journals

Academic and behavior combined support: A single-case practice-based replication study

Conceptualizing and Evaluating Replication Across Domains of Behavioral Research

Research on Teaching: Opening the Door to Special Education Classrooms

Publishing Single-Case Research Design Studies That Do Not Demonstrate Experimental Control

Giving Away the Science of School Psychology

Reproducible Research: A Retrospective