Abstract:This study provides a systematic review of the methodological features of meta-analyses in second language learning. The synthesis aims to inform how meta-analyses in L2 learning have been conducted, evaluate whether methodological decisions are aligned with norms and standards, identify issues, and suggest solutions based on expert advice and statistical guides. A total of 120 meta-analyses were retrieved and coded for key features related to bibliographic and demographic characteristics, search and selection, publication bias, quality control, data coding, data analysis, and effect size interpretation. The synthesis showed that 98 meta-analyses examined the effectiveness of instructional treatments, 21 investigated correlations, and one explored the occurrence of events. These meta-analyses included an average of 37 primary studies (range = 9–302). Common selection criteria the meta-analyses applied included publication type, availability of data for effect size calculation, learner traits, learners' target languages, publication dates, publication language, independent variables, dependent variables, and so on. Major strategies used to detect publication bias included creating a funnel plot, using trim-and-fill analysis, and calculating a fail-safe N. Typical moderators examined in the meta-analyses related to research context, treatment features, sample characteristics, and outcome measures. The synthesis also identified a number of issues, including failure to report key features such as model selection (fixed- vs. random-effects model), effect size weighting, and so on; conducting moderator analysis based on very small cell sizes (e.g., only one study in a subgroup); lack of justification for certain methodological decisions such as using d instead of g, using confidence intervals rather than Q-tests to identify significant moderators; lack of quality control; and confounding study-based and synthesis-based moderators. We offer advice on identified issues, and call for more transparency of reporting.

Meta-analysis in Second Language Research: Choices and Challenges

A systematic review of meta-analyses in second language research: current practices, issues, and recommendations

Domain definition and search techniques in meta-analyses of L2 research (Or why 18 meta-analyses of feedback have different results)

Applying meta-analysis to research on bilingualism: An introduction

Second-order synthesis of meta-analytic studies in applied linguistics (1998–2021)

A scoping review of research on languaging in second language education

A Decade of Research Into Second Language Acquisition: A State-of-the-Science Review (2009-2019)

A Call For Cautious Interpretation Of Meta-Analytic Reviews

A Meta-Synthesis of Empirical Research on the Effectiveness of Computer-Mediated Communication (CMC) in SLA.

A Hands-On Tutorial for Systematic Review and Meta-Analysis With Example Data Set and Codes

The role of research design in the reproducibility of L1 and L2 language networks: A review of bilingual neuroimaging meta-analyses

Analysis of International Research Trends on Literacy in Second Language Education: Focusing on Foreign Studies on ESL and EFL

Fragile Findings, Complex Conclusions, and Meta-Analysis in Special Education

A Review Of Second Language Teaching And Learning Through Short Fiction And Communicative Approach

A Review of Meta-Analyses of Correlation Coefficients on L2 Reading Comprehension

Task Planning and Oral L2 Production: A Research Synthesis and Meta-analysis

Meta-analysis in information systems research: method choices and recommendations for future research

Language learning strategies research in English as foreign language contexts: A systematic literature review

Single-case design meta-analyses in education and psychology: a systematic review of methodology

A Meta-analysis of the Effectiveness of Second Language Pragmatics Instruction

Second and Foreign Language Vocabulary Learning through Digital Reading: A Meta-Analysis