Abstract:AbstractBackgroundLanguage assessment plays a pivotal role in language education, serving as a bridge between students' understanding and educators' instructional approaches. Recently, advancements in Artificial Intelligence (AI) technologies have introduced transformative possibilities for automating and personalising language assessments.ObjectivesThis article aims to explore the design and implementation of AI‐enabled assessment tools in language education, filling the research gaps regarding the impact of assessment type, intervention duration, education level, and first language learner/second language learner (L1/L2) on the effectiveness of AI‐enabled assessment tools in enhancing students' language learning outcome.MethodsThis study conducted a systematic review and meta‐analysis to examine 25 empirical studies from January 2012 to March 2024 from six databases (including EBSCO, ProQuest, Scopus, Web of Science, ACM Digital Library and CNKI).ResultsThe predominant design in AI‐driven assessment tools is the structural AI architecture. These tools are most frequently deployed in classroom settings for upper primary students within a short duration. A subsequent meta‐analysis showed a medium overall effect size (Hedges's g = 0.390, p < 0.001) for the application of AI‐enabled assessment tools in enhancing students' language learning, underscoring their significant impact on language learning outcomes. This evidence robustly supports the practical utility of these tools in educational contexts.ConclusionsThe analysis of several moderator variables (i.e., assessment type, intervention duration, educational level and L1/L2 learners) and potential impacts on language learning performance indicates that AI‐enabled assessment could be more useful in language education with a proper implementation design. Future research could investigate diverse instructional designs for integrating AI‐based assessment tools in language education.

Review of recent empirical research (2011–2018) on language assessment in China

Review on the validity of China’s Standards of English Language Ability

Assessing Second Language Listening over the Past Twenty Years: A Review Within the Socio-Cognitive Framework

Assessing Language Testing: A Corpus-Based Approach

Bridging Local Needs and National Standards: Use of Standards-Based Individualized Feedback of an In-House EFL Listening Test in China

A review on empirical research of linguistic relativity in recent years

The teaching of Chinese as a second or foreign language: a systematic review of the literature 2005–2015

A Systematic Review and Meta‐analysis of AI‐enabled Assessment in Language Learning: Design, Implementation, and Effectiveness

Research Trends and Development Patterns in Language Testing Over the Past Three Decades: A Bibliometric Study

The Effect of a National Education Policy on Language Test Performance: a Fairness Perspective

Teaching Chinese as a second or foreign language to non-Chinese learners in mainland China (2014–2018)

Researching English language textbooks: A systematic review in the Chinese context (1964–2021)

A Review of the Research on Language Variation in China in Recent 20 Years

Research in foreign language teaching and learning in China (2012–2021)

Recent Development of English Pronunciation Assessment and Testing Studies：World Trends and Their Messages for the Teaching in China

A scoping review of the literature on content and language integrated learning assessment

Review of Research on Learning and Instruction with Specific Reference to Reading Chinese as an Additional Language (1976–2018)

A Systematic Review of the Research on Chinese Character Teaching and Learning

Critical discourse studies eleven years on in China (from 2011 to 2021): A critical review

Literature Review on Language Learning Strategy Research in China

A Systematic Review of Chinese Language Learning Strategies in the Past Decade (2011-2020)