Review of the Japanese-Language Proficiency Test

Hitoshi Nishizawa,Daniel R. Isbell,Yuichi Suzuki

DOI: https://doi.org/10.1177/02655322221080898

2022-03-09

Language Testing

Abstract:Language Testing, Ahead of Print.

linguistics

What problem does this paper attempt to address?

Review of Carroll (2001): Language Planning and Language Change in Japan

Haitao Liu

DOI: https://doi.org/10.1075/lplp.29.2.09liu

2005-01-01

Abstract:Preview this article: Tessa Carroll. Language Planning and Language Change in Japan, Page 1 of 1 < Previous page | Next page > /docserver/preview/fulltext/lplp.29.2.09liu-1.gif
Providing Test Performance Feedback That Bridges Assessment and Instruction: The Case of Two Standardized English Language Tests in Japan

Yasuyo Sawaki,Rie Koizumi

DOI: https://doi.org/10.1080/15434303.2017.1348504

IF: 2.143

2017-07-03

Language Assessment Quarterly

Abstract:This small-scale qualitative study considers feedback and results reported for two major large-scale English language tests administered in Japan: the Global Test of English Communication for Students (GTECfS) and the Eiken Test in Practical English Proficiency (Eiken). Specifically, it examines current score-reporting practices in student and teacher score reports and supplementary materials as well as students’ and teachers’ perceptions and uses of reported information. First, to summarize key features of performance feedback provided, two independent judges coded the information that was included as feedback in the score reports and supplementary materials as well as how the feedback was presented for both tests, based on Roberts and Gierl’s (2010) test score report analysis framework. Second, 16 students and five teachers at three upper secondary schools (Grades 10–12) in Japan were interviewed to explore how they perceived the assessment results and used them in their learning and teaching. Key study results showed that the documents generally followed principles of good score-reporting practice suggested in previous studies, while student and teacher interviews identified various issues to be considered to improve the communication of test performance feedback to its intended audiences.

linguistics,psychology, educational
A comparative analysis of language choices in written peer feedback provided by high- and low-proficiency Japanese EFL students

Sugene Kim

DOI: https://doi.org/10.1177/13621688231193074

IF: 3.401

2023-09-21

Language Teaching Research

Abstract:Language Teaching Research, Ahead of Print. This study reports on part of a larger research project that investigates the role of students' first language in learning English as a foreign language (EFL) in the context of Japanese higher education. Despite the prevalent use of peer review in EFL writing classes, students' use of different languages and the effects of English proficiencies on collaborative interaction as mediated in written peer feedback remain underresearched. To add to the rather sparse literature, this study comparatively analysed peer feedback provided by high- and low-proficiency Japanese EFL students and conducted stimulated recall interviews to determine the factors affecting their language choices. The text analysis presented a stark contrast in students' language choices stemming mainly from a difference in the type of feedback provided by each group: 'corrective feedback' versus 'complimentary overall comment'. The stimulated recall interview accounts suggested that this polarization is attributable mostly to the proficiency gap between peer dyads and the affective need to contribute to the peer-review process. Inconsistent with the reports of previous studies that peer review benefits all students involved, the advanced-level participants voiced serious doubts about its effectiveness.

education & educational research,linguistics
A Computer-Assisted Tool for Automatically Measuring Non-Native Japanese Oral Proficiency

Wenchao Li,Zhentao Zhong,Haitao Liu

DOI: https://doi.org/10.1080/09588221.2024.2368838

IF: 7

2024-01-01

Computer Assisted Language Learning

Abstract:This study presents the development of an automated tool for assessing the speech of non-native Japanese speakers. The tool integrated an accent-trained automatic speech recording system based on deep neural networks, audio-text transcription, and natural language processing technologies to perform annotation, analysis, calculation, and scoring. A dataset consisting of audio recordings from 1000 Japanese learners across 16 countries was used for machine training. Validity and reliability tests were parallelly developed to evaluate: (a) the measures' ability to distinguish test takers with different oral proficiency levels; (b) the agreement between the annotations and scoring performed by the tool and human raters. The results, obtained through precision, recall, F-Score, quadratically-weighted kappa, proportional reduction of mean squared error, Pearson correlation, and multi-faceted Rasch measurement, demonstrated a relatively high reliability of the automated tool. The tool provides users with both a holistic score and individual scores for each criterion, allowing them to gain insights into their overall performance as well as detailed evaluations for each aspect of their proficiency. Consequently, it can be utilized for employment, education, and self-learning purposes.
Assessing the Interactional Competence in English Speaking of Japanese Junior High School Students in an Interview Test

Kazuyo KAWAMURA,Osamu TAKEUCHI

DOI: https://doi.org/10.20622/jltajournal.26.0_3

2023-01-01

JLTA Journal

Abstract:According to a large-scale survey administered by the Ministry of Education, Culture, Sports, Science and Technology, Japan, in 2021, interview tests wherein students speak with a teacher or an assistant language teacher are mainly used in Japanese junior high schools to assess interactional competence in speaking English. However, there has been a lack of investigation regarding the types of utterances elicited by students during interview tests conducted by teachers. This study aimed to fill this gap by involving a teacher who interviewed 41 junior high school students, allowing for a holistic and analytical examination of the dialogues during the interviews. The analysis of these student-teacher dialogues revealed that the teacher actively assisted students in continuing the dialogue by offering new topics, developing them, and asking topic-related questions to continue the dialogue. The authors argue that these findings indicate that the interviews demonstrated the teacher's interactional competence more than that of the students, making it difficult to assess students' interactional competence through teacherconducted interviews. Therefore, alternative speaking-test formats are needed to assess the students' interactional competence in English.
English Language Teaching and Learning in Japan: History and Prospect

Yoshifumi Saito

DOI: https://doi.org/10.1007/978-981-13-2632-5_13

2019-01-01

Abstract:The history of English language teaching (ELT) in Japan for the last hundred years is quite often depicted, deploringly as well as deplorably, as one of constant failure and confusion. This is largely because Japanese people on average tend to stay toward the lowest end of English proficiency on some international scales (including TOEFL and IELTS), most of them failing to reach a level they find satisfactory despite the time and energy they spend on learning the language. On the assumption that their time and energy have been misspent and that Japan has taken wrong approaches to ELT, a considerable number of progressive scholars and teachers proposed, at every turn of the abovementioned history, various innovations, most conspicuously, of teaching methodologies. The methodological innovations have mostly taken the form of introducing West-born methods and approaches, each of which, however, in turn has proved or is almost sure to prove not as effective as it first looked. The same assumption has also urged political or educational dignitaries to propose more administration-based “root-and-branch” innovations, only to aggravate the complications of classroom ELT situations all over Japan.
A content analysis of prefectural senior high school entrance exams: What aspects of English do they assess and how?

Peter O'Keefe,David Allen,,

DOI: https://doi.org/10.37546/jaltsig.teval26.2-1

2022-12-18

Abstract:Research into high-stakes English tests in Japan has typically focused on entrance examinations to university while few studies have investigated those for high school. In this study, a content analysis was conducted of 24 prefectural English entrance examinations for public high schools administered in 2022. We focused on the language skills and subskills targeted in the exams by referring to the socio-cognitive framework (Weir, 2005), and examined the item types and response formats, and readability of the reading passages. Key findings revealed that in terms of item distribution, the exams focus primarily on reading (62.38%), followed by listening (29.16%), and finally writing (8.46%). In terms of subskills, the exams tended to target comprehension of main ideas at the text level rather than comprehension of specific details or discrete linguistic knowledge at the sentence level. Overall, the findings highlight the convergence between the exam content and the goals of the MEXT course of study. However, the dominant focus on receptive skills remains, which presents a potential barrier to achieving positive washback from these high- stakes exams.
Development of a Screening Test for the Early Identification of Japanese Children with Difficulties Learning English: A Preliminary Study

Sayano Kamioka,Ikuyo Ishizaka,Keita Suzuki,Yuki Hara,Wakana Hata,Yoko Mizuto

DOI: https://doi.org/10.2490/prm.20220038

2022-07-26

Abstract:Objectives: In Japan, there is no established method to assess the ability to read and write in English. To address this problem, we sought to develop a screening test for the early detection of students who show difficulties in reading and writing in English. Methods: The participants were 425 fifth- and sixth-grade elementary school students and 526 first- through third-grade junior high school students. While setting up the task items, we focused on the assessment of visual information processing ability related to letter-symbol information processing. Q1 was a letter identification task, Q2 was a letter recognition task, Q3 was a discrimination task, Q4 was a lexical decision task, Q5 was a semantic comprehension task, Q6 was a meaningful sentence copy task, and Q7 was a nonsensical sentence copy task. Q1 to Q5 assessed reading ability and Q6 and Q7 assessed writing ability. Results: The comparison of basic distribution between elementary and junior high school showed that there were differences in the distribution of both reading and writing scores between the two school types (P<0.05). At the cut-off value of -1.5 SD, 7.8% of the students were extracted for reading scores and 4.2%-5.5% for writing scores. Conclusions: The extraction rate of students using this screening test supports the results of previously published studies. Thus, this screening test is considered suitable for identifying elementary and junior high school students who face difficulties in reading and writing in English.
Assessing Language Testing: A Corpus-Based Approach

Youxia Sun,Lianzhen He

2015-01-01

Abstract:Language Testing has now completed three decades of publication, during which a great amount of significant work has been done. In this paper, a corpus of 30 volumes of this journal (1984-2013) is built to analyze the research foci and changes, research methodologies, representative scholars and research centers that have long enjoyed cachet. Based on the quantitative and qualitative analysis of these data, it can be concluded that this journal is representative of the development in the field of language testing, and it has been and is reflecting and promoting research, as well as changes in this field and beyond over the years.
The roles of language proficiency and study abroad in Japanese students’ receptive pragmatic competence

Natsuko Shintani,Rod Ellis

DOI: https://doi.org/10.1075/ap.20011.shi

2022-02-09

Applied Pragmatics

Abstract:Abstract This study investigated the pragmatic competence of 50 Japanese English major students, divided into two groups, one of which participated in a study abroad (SA) program and the other stayed at home (AH) taking regular university classes. Two tests were used to measure two aspects of the students’ pragmatic competence. The Irony Test measured their ability to identify negative and positive irony. The Metapragmatic Test measured their ability to identify inappropriate speech acts and their understanding of why they were inappropriate. Results showed that, compared with a group of native speakers, the students had difficulty in identifying both irony – especially positive irony – and speech act inappropriateness. The students’ language proficiency was related to their metapragmatic ability but not to their ability to detect irony. A comparison of the SA and AH students revealed a small advantage for the former in the Metapragmatic Test but not in the Irony Test.
A study of similarity and/or dissimilarity of brain activation of Japanese learners during written tests and conversation

Yu Hirata

DOI: https://doi.org/10.21820/23987073.2021.2.70

2021-02-26

Impact

Abstract:In modern-day language teaching, ensuring that there is a practical communicative proficiency is the first priority, rather than trying to enhance a learner's grammatical knowledge. This can be said to be true in a first language, as well as when somebody attempts to learn a second language. Researchers are now exploring ways that written tests can be used to assess a learner's speaking ability or communicative proficiency. Dr Yu Hirata is based within the Graduate School of Language Education and Information Science at Ritsumeikan University in Japan, and is heading up a study of the similarities and dissimilarities of brain activity in Japanese learners during written tests and conversation, which could provide significant benefits to those teaching and studying language courses.
Preparing for the Speaking Tasks of the TOEFL iBT® Test: An Investigation of the Journeys of Chinese Test Takers

Guoxing Yu,Lianzhen He,Pauline Rea-Dickins,Richard Kiely,Yanbin Lu,Jing Zhang,Yan Zhang,Shasha Xu,Lin Fang

DOI: https://doi.org/10.1002/ets2.12145

2017-01-01

ETS Research Report Series

Abstract:Language test preparation has often been studied within the consequential validity framework in relation to ethics, equity, fairness, and washback of assessment. The use of independent and integrated speaking tasks in the TOEFL iBT® test represents a significant development and innovation in assessing speaking ability in academic contexts. Integrated tasks that involve synthesizing and summarizing information presented in reading and listening materials have the potential to generate new test preparation strategies. This study investigated the experiences of over 1,500 Chinese test takers and 23 teachers who were preparing for the TOEFL iBT speaking tasks. It examined the frequency of use of a number of different test preparation activities and materials, reasons, and expectations for taking preparation courses and the features of preparation courses. In addition, we examined the usefulness of test preparation from two perspectives: studentsu0027 and teachersu0027 perceptions as well as the relationship between test preparation and performance. Data were collected via questionnaires, focus group discussions, interviews with test takers and teachers, and classroom observations. The data showed that (a) test preparation was a hugely complex, multiple-components construct, and teaching and learning test-taking strategies compose the most prominent feature of intensive preparation courses; (b) there were significant age-related differences in studentsu0027 preparation activities and focuses, although with small effect sizes; (c) there was a high agreement between teachers and students in their views on the usefulness of test preparation activities; and (d) there existed only a weak relationship between test preparation and performance. The only significant predictor of studentsu0027 test performance was the frequency of their use of the TOEFL Practice Online TPO® practice tests. The findings of the study can enhance our understanding of the pedagogical practices that characterize test preparation programs and contribute to the ongoing validity argument for the TOEFL iBT Speaking test. The implications of the findings for test publishers, test takers, teachers, and test preparation schools are discussed with reference to the instructional, learning, and affective aspects of the multifaceted construct of test preparation.
Beyond the threshold: Exploring English language proficiency, linguistic challenges, and academic language skills of Japanese students in an English medium instruction programme

Ikuya Aizawa,Heath Rose,Gene Thompson,Samantha Curle

DOI: https://doi.org/10.1177/1362168820965510

IF: 3.401

2020-10-24

Language Teaching Research

Abstract:This article examines the relationship between Japanese undergraduate students’ English language proficiency and English language-related challenges faced when studying an international business course through English. It also examines English language proficiency thresholds students need to reach in each academic skill (i.e. reading, listening, speaking and writing) to experience a lower level of linguistic challenges. A total of 264 students were surveyed in Tokyo, Japan, and 13 follow-up interviews were conducted. Exploratory and confirmatory factor analyses confirmed the underlying factors in the EMI (English medium of instruction) Challenges Scale loaded onto a priori assumptions of dimensions falling along skill-based constructs. Analysis of questionnaire data revealed that English language proficiency (i.e. TOEIC score) was a statistically significant predictor of challenges in the EMI programme. While no clear discernible threshold was observed, the differences in perceived ease of study at different levels of English proficiency influenced the challenges students reported for each academic skill. Interview data uncovered the multi-faceted nature of how the thresholds are determined not only by language proficiency but also by other factors, such as prior content knowledge, motivation, and the classroom learning environment. Practical implications for pedagogy are also discussed.

education & educational research,linguistics
Students' Learning Autonomy: A Case Study of Undergraduate Course of Japanese Language Program

Huang Liyuan

DOI: https://doi.org/10.1007/s10936-023-09992-x

Abstract:This research aimed to determine the impact of the effectiveness in the application of an independent method of teaching students on their progress in achieving a certain level of Japanese. The study used key methods that implied the experimental involvement of 60 3rd- and 4th-year American and Chinese students with the N3 and N2 levels of Japanese to pass a simplified version of the international JLPT (Japanese Language Proficiency Test) after applying the autonomy learning methodology. The analysis and processing of the experimental exam results were performed using the Microsoft Excel program and derivation of the average statistical percentage of the number of points scored at each stage of the N3 and N2 exams. The results of the study and the test scores led to the conclusion that the most difficult tasks for American students are those in writing and reading. On the other hand, Chinese students have difficulties in listening and speaking Japanese over linguistic phonetic differences and complex grammar compared to Chinese. Referring to the N2 group of American and Chinese students, their scores are somewhat equal due to the total complexity of this level, which requires utmost care for all four skills of foreign language learning. The present findings can serve as auxiliary material for the educational sphere in terms of an individual approach in autonomy learning to more effectively study Japanese and obtain positive results when passing the JLPT test.
Assessing Japanese junior high school students’ English achievement through computer-based testing in the classroom: a case of integrated reading-into-writing continuous task

Noriyasu Niimi,Nobukazu Matsuura

DOI: https://doi.org/10.1186/s40468-022-00189-y

2022-09-13

Language Testing in Asia

Abstract:Abstract Introduction This paper describes the exploratory case and initial evaluation of the computer-based testing (CBT) prototype. The advantage of CBT over paper-based testing (PBT) is that it allows us to control the order of questions and provides test takers with continuous tasks capturing their thought processes. Additionally, their response process data such as response time (RT) can be obtained. Taking advantage of these, we created a CBT prototype in the classroom for Japanese junior high school students. Case description A CBT model was created to assess integrated reading and writing ability and was administered to 32 junior high school students. Their process achievement and the relation between the process response and writing quality were analyzed. Students’ RT for each screen was analyzed using hierarchical cluster analysis. Discussion and evaluation We identified not only students facing difficulties at each stage of a series of thought processes but also five clusters that include students spending too much time reading source texts or organizing their ideas. We suggest how CBT can be developed to identify students with difficulties and applied to teaching. Conclusions CBT has the possibility of detecting students who are able to complete the language performance task by controlling the order of answers, asking questions sequentially, and obtaining RT effectively.

education & educational research,linguistics
Awareness of English as an International Language: The Impact of Activities in English Learning Facilities in Japan

DOI: https://doi.org/10.55057/ijares.2023.5.4.9

2023-12-31

International Journal of Advanced Research in Education and Society

Abstract:This study investigates the effects of short-term study abroad and cross-cultural activities on the English proficiency of Japanese university students with little to no international exposure. The results confirm the effects on their international outlook, cross-cultural understanding, and motivation to learn English. Their views of language—particularly their attitude toward English as an international language—are also examined. Due to external factors, such as the cancellation of overseas training programs after the spread of COVID-19, we took selected students to two English language study facilities in Japan: Tokyo Global Gateway (TGG)[1] and British Hills (BH)[2]. The students spent one day in TGG and two days in BH, both of which had foreign instructors, to examine how their views of language changed after experiencing several activities in English. We also examined how the degree of change varied depending on the facility. The questionnaire consisted of 10 questions on language perspectives. The results show a decrease in the belief that one must necessarily study in Japan, a decrease in the psychological pressure of feeling obliged to speak like a native, and an increase in the desire to speak like a native. The changes in the students’ views also differed qualitatively, depending on the characteristics of the facilities.
Equivalency of Picture-Based Speaking Tasks: An Investigation of Complexity, Accuracy, Lexis, and Fluency

Joe Kakitani,

DOI: https://doi.org/10.37546/jalttlt47.2-1

2023-03-01

The Language Teacher

Abstract:In experimental research, a pretest–posttest design is often used to examine the effect of treatment or intervention. Establishing equivalency across tests is essential for this type of research to ensure the validity of the study results. However, studies that explore test/task equivalency are scarce in second language research. This study investigates the equivalency of seven picture-based narrative tasks such as those commonly used in second language research and language tests. The oral performances elicited from 20 Japanese university students were analyzed in terms of their complexity, accuracy, lexis, and fluency. Despite controlling for the task-related variables of story length, sequential structure, and storyline complexity, the results were similar only in terms of fluency and not with regard to complexity, accuracy, or lexis. These findings suggest that it is important to determine the equivalency of testing materials as seemingly similar tasks do not necessarily elicit similar oral performances. 実験的研究では、対処や介入の効果を調べるために事前・事後テストデザインがよく用いられる。この種の研究では、結果の妥当性を担保するためにテストの同等性を確立することが重要である。しかし、第二言語研究において、テストやタスクの同等性を調査した研究は極めて少ない。本研究は、第二言語研究や言語テストにおいて頻繁に使用されている写真描写タスクに焦点を当て、7つのタスクの同等性を検証する。そのために、20人の日本人大学生のスピーキングパフォーマンスを、複雑さ、正確さ、語彙、流暢さの尺度を使って分析した。結果は、タスクに関わる要因である物語の長さ、構造、複雑さを統制したにもかかわらず、流暢さにおいてのみタスクの類似性が示され、複雑さ、正確さ、語彙においては差異が認められた。本研究結果は、外見的には似ているタスクであっても同等のスピーキングパフォーマンスに必ずしも結びつかないことを示し、テストに使用する題材の同等性を検証することの重要性を示唆している。
Proposing change in university entrance examinations: A tale of two metaphors

David Allen

DOI: https://doi.org/10.37546/jaltsig.teval24.2-2

2020-12-01

Abstract:This article describes a recent education reform initiative concerning English education in Japan, specifically the proposed introduction of four-skills tests as part of the university entrance admissions process. The first aim is to summarize, in English, some of the key issues and events concerning the reform. To this end, background information and a timeline of key events since 2016 is provided. The second aim is to contrast proposals made by two academic organizations, the Japan Language Testing Association (JLTA) and the Science Council of Japan: Language and Literature Committee (SCJ). It is shown that, while agreeing on a number of specific issues related to the reform, these two organizations take starkly different positions in terms of their general orientation, which, it is argued, reflects the background of the organizational members and their views on foreign language education in Japan. These contrasting positions are discussed with reference to the metaphor, to throw the baby out with the bathwater. Finally, it is argued that a number of criticisms levelled at the proposed use of private four-skills tests illustrate a reluctance to engage with issues related to the currently used university entrance exams; in other words, these criticisms are made while ignoring the elephant in the room.
Language testing

DOI: https://doi.org/10.1017/s0261444802241816

2002-07-01

Language Teaching

Abstract:02–517 Brindley, Geoff (Macquarie U., New South Wales, Australia; Email: gbrindle@laurel.ocs.mq.edu.au). Outcomes-based assessment in practice: Some examples and emerging insights. Language Testing (London, UK), 18, 4 (2001), 393–407.

education & educational research,linguistics
Testing Virtual Reality for Eliminating Japanese University Students' English-Speaking Anxiety: Cases of International Conference and Restaurant

Tsukasa Yamanaka,Yasushi Miyazaki,Syuhei Kimura,Miho Yamashita,Yukie Kondo,Atsuko K. Yamazaki,Kayoko H. Murakami

DOI: https://doi.org/10.18178/ijlt.9.1.49-55

2023-01-01

International Journal of Learning and Teaching

Abstract:Reflecting COVID-19 outbreak that prevents educators from making close contacts with students, digitalization of language teaching is gradually getting educators’ attention. With respect to this situation, this research conducted an experiment of using Virtual Reality (VR) materials for 69 English as a STEM-major Foreign Language (EFL) learners at a Japanese university. Polishing usability of Virtual Reality (VR) English language education materials could improve the learners’ performance of English language use. However, careless use of VR materials could fail, unless the educators consider its technical and pedagogical limitations. The experiment used pairs of Virtual reality headset with two scenarios (international conference and restaurant). From pre-post experiment surveys on speaking anxiety scale, students with higher Test of English for International Communication (TOEIC) listening scores showed more anxiety on speaking. Also, post-experiment open-ended questionnaires suggested that usability of VR materials should be improved.

Review of the Japanese-Language Proficiency Test

Review of Carroll (2001): Language Planning and Language Change in Japan

Providing Test Performance Feedback That Bridges Assessment and Instruction: The Case of Two Standardized English Language Tests in Japan

A comparative analysis of language choices in written peer feedback provided by high- and low-proficiency Japanese EFL students

A Computer-Assisted Tool for Automatically Measuring Non-Native Japanese Oral Proficiency

Assessing the Interactional Competence in English Speaking of Japanese Junior High School Students in an Interview Test

English Language Teaching and Learning in Japan: History and Prospect

A content analysis of prefectural senior high school entrance exams: What aspects of English do they assess and how?

Development of a Screening Test for the Early Identification of Japanese Children with Difficulties Learning English: A Preliminary Study

Assessing Language Testing: A Corpus-Based Approach

The roles of language proficiency and study abroad in Japanese students’ receptive pragmatic competence

A study of similarity and/or dissimilarity of brain activation of Japanese learners during written tests and conversation

Preparing for the Speaking Tasks of the TOEFL iBT® Test: An Investigation of the Journeys of Chinese Test Takers

Beyond the threshold: Exploring English language proficiency, linguistic challenges, and academic language skills of Japanese students in an English medium instruction programme

Students' Learning Autonomy: A Case Study of Undergraduate Course of Japanese Language Program

Assessing Japanese junior high school students’ English achievement through computer-based testing in the classroom: a case of integrated reading-into-writing continuous task

Awareness of English as an International Language: The Impact of Activities in English Learning Facilities in Japan

Equivalency of Picture-Based Speaking Tasks: An Investigation of Complexity, Accuracy, Lexis, and Fluency

Proposing change in university entrance examinations: A tale of two metaphors

Language testing

Testing Virtual Reality for Eliminating Japanese University Students' English-Speaking Anxiety: Cases of International Conference and Restaurant