Abstract:Constructing arguments is essential in science subjects like chemistry. For example, students in organic chemistry should learn to argue about the plausibility of competing chemical reactions by including various sources of evidence and justifying the derived information with reasoning. While doing so, students face significant challenges in coherently structuring their arguments and integrating chemical concepts. For this reason, a reliable assessment of students' argumentation is critical. However, as arguments are usually presented in open‐ended tasks, scoring assessments manually is resource‐consuming and conceptually difficult. To augment human diagnostic capabilities, artificial intelligence techniques such as machine learning or natural language processing offer novel possibilities for an in‐depth analysis of students' argumentation. In this study, we extensively evaluated students' written arguments about the plausibility of competing chemical reactions based on a methodological approach called computational grounded theory. By using an unsupervised clustering technique, we sought to evaluate students' argumentation patterns in detail, providing new insights into the modes of reasoning and levels of granularity applied in students' written accounts. Based on this analysis, we developed a holistic 20‐category rubric by combining the data‐driven clusters with a theory‐driven framework to automate the analysis of the identified argumentation patterns. Pre‐trained large language models in conjunction with deep neural networks provided almost perfect machine‐human score agreement and well‐interpretable results, which underpins the potential of the applied state‐of‐the‐art deep learning techniques in analyzing students' argument complexity. The findings demonstrate an approach to combining human and computer‐based analysis in uncovering written argumentation.

A School Student Essay Corpus for Analyzing Interactions of Argumentative Structure and Quality

Analyzing Critical Thinking Elements in the Argument Structure of Non-English-Major Chinese Undergraduate Students' Writing

A Corpus for Argumentative Writing Support in German

Analyzing Culture-Specific Argument Structures in Learner Essays.

Mind the Gap: Automated Corpus Creation for Enthymeme Detection and Reconstruction in Learner Arguments

Building an Annotated L1 Arabic/L2 English Bilingual Writer Corpus: The Qatari Corpus of Argumentative Writing (QCAW)

An Exploratory Study of Argumentative Writing by Young Students: A Transformer-based Approach

Argumentation Mining on Essays at Multi Scales.

Argument Quality Assessment in the Age of Instruction-Following Large Language Models

Rhetoric, Logic, and Dialectic: Advancing Theory-based Argument Quality Assessment in Natural Language Processing

Assessing the Quality of Argumentative Texts: Examining the General Agreement Between Different Rating Procedures and Exploring Inferences of (Dis)agreement Cases

CityU corpus of essay drafts of English language learners: a corpus of textual revision in second language writing

Creating a Domain-diverse Corpus for Theory-based Argument Quality Assessment

The Discussion Tracker Corpus of Collaborative Argumentation

Exploring new depths: Applying machine learning for the analysis of student argumentation in chemistry

Towards an Argument Mining Pipeline Transforming Texts to Argument Graphs

Structuring written arguments in primary and secondary school: A systemic functional linguistics perspective

Automated Evaluation for Student Argumentative Writing: A Survey

Is Argument Structure of Learner Chinese Understandable: A Corpus-Based Analysis

CEAMC: Corpus and Empirical Study of Argument Analysis in Education Via LLMs

Chinese Argumentative Writing Quality Evaluation Based on Multi-perspective Modeling