Abstract:The seminal work by Ericsson and Simon established verbal reports as a genuine way to get a glimpse into the “black box” of the workings of the human mind, and it is now recognised as a method of value in the pursuit of new knowledge relating to, for instance, learner strategies, cognitive strategies, human-computer interaction, and functionality of software and hardware. The present paper presents a think-aloud study (concurrent verbal report) which was conducted with university students of English as a Foreign Language, who were using a CALL program to learn English syntax. The students were asked to verbalise their thoughts while conducting the task of parsing sentences and building tree diagrams on the computer. The computer program Visual Interactive Syntax learning (VISL) is operational in fifteen different languages ranging from English, French, and German to Arabic, Danish, Dutch, Latin, Greek, and Japanese, and allows the students to learn and work with syntax in their own speed. VISL contains a large pre-tagged corpus but also allows for free text input. The tree diagrams build on a strict division of form and function, and this is supported by the colour scheme of the interface. The purpose of the think-aloud (TA) study was to gain direct access—as far as that is possible—to the mental processes and strategies which the students employ in their use of the software in order to better understand the learning strategies they made use of in these processes. The study and the recordings took place in two phases: an initial recording at the beginning of the experiment and a later, second recording after a 10-week period of weekly sessions of working with the software. This study was intended to supply data that could supplement and illuminate quantitative data which were measured in a pre-test/post-test formula, in a design which included a control group. In the initial recordings there were some critical remarks relating to the software but these were almost completely absent from the second recording. An interesting aspect is that despite the fact that the content of this learning process—English syntax—continued to create problems for many of the participating students, this was not projected into dissatisfaction with the software; quite the contrary. The think-aloud protocols were analysed for information on strategies and human-computer interaction. The new information from this study is that it appears that the computer is able to further the qualities in the students which are desirable in solving the learning task. At the outset of the experiment the low-achieving students had very few cognitive strategies, but a high rate of affective strategies. During the time of the training these students developed the more appropriate strategies for the task. In other words, the computer (and/or the software) helped students develop a better strategy for learning, which leads to the question of whether teaching of strategy or awareness-raising exercises related to appropriate strategies would lead to higher learning outcomes if initiated from the beginning of the instructional efforts.

The effect of cognitive load on a statistical dialogue system

Statistical Methods for Building Robust Spoken Dialogue Systems in an Automobile

Effects of speech- and text-based interaction modes in natural language human-computer dialogue

Towards Measuring the Cognitive Loads of Different Dialog Acts Through Dependency Distance.

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

User Study of the Bayesian Update of Dialogue State Approach to Dialogue Management

Spoken Dialog Challenge 2010: Comparison of Live and Control Test Results.

Effect of cognitive load on drivers' State and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control

CHAT: a Conversational Helper for Automotive Tasks

Estimation of the Cognitive Functioning of the Elderly by AI Agents: A Comparative Analysis of the Effects of the Psychological Burden of Intervention

Cognitive Load and Productivity Implications in Human-Chatbot Interaction

Mitigating Information Overload in e-Commerce Interactions with Conversational Agents

Dialogue Management Systems: a Survey and Overview

Computer-aided analysis and design for spoken dialogue systems based on quantitative simulations

Decision Making Strategies for Finite-State Bi-automaton in Dialog Management

Characterisation of Cognitive Load Using Machine Learning Classifiers of Electroencephalogram Data

Evaluating Older Users' Experiences with Commercial Dialogue Systems: Implications for Future Design and Development

Dialogos: a robust system for human-machine spoken dialogue on the telephone

The Effect of Cognitive Load on Auditory Susceptibility During Automated Driving

Towards Best Experiment Design for Evaluating Dialogue System Output

Cognitive processes at work in CALL