Abstract:Given the widespread concern on collaborative problem solving (CPS) skills, there has been an increasing interest in the last few years to explore how to assess them with digital technologies. This study systematically reviewed how CPS skills have been assessed with digital technologies in the literature. A total of 40 articles were reviewed to analyze specific computer-based assessment instruments of CPS skills from four perspectives: research context, theoretical model for assessment, assessment type, and reliability and validity evidence. The results indicate that most tests target a sample of less than 500 junior students. Nine theoretical models are employed for assessing CPS skills, most of which treat these skills as an explicit combination of social and cognitive skills and are applied to a limited range of participants' age levels, collaboration features, and team compositions. A total of 22 tests have been employed and fallen into four types, i.e., the ones with specific predefined messages in human-agent mode, and those with online chat box, videoconferencing, and face-to-face collaboration in human-human mode. Each type of these tests demonstrates great diversities in participants' age levels, types of CPS task(s), team compositions, types of assessment data, and methods of data recording and scoring. A certain number of tests lack reliability and validity evidence. Our findings are expected to benefit relevant researchers and test developers in terms of providing suggestions for future research which include testing the applicability of theoretical models for assessing CPS skills across a wide range of assessment contexts. In addition, future researchers should improve the development, data processing, and report of those four types of computer-based assessment instruments of CPS skills through different approaches, respectively.

Research on Automatic Assessment of Transferable Skills.

Adaptive Stage-Aware Assessment Skill Transfer for Skill Determination

Adaptive online training and evaluation systems

An Overview of Approaches to the Automated Assessment of Subjective Tests

The Development of an Automatic Test Assembly System for a Formative Assessment in Mastery Learning Instruction: Case of the SQL Mastery Course

Research on the Human-aided Auto-assessment for Translation Tests in College English

A Method of Computer Automatic Scoring for Subjective Questions

A Web-Based Examination and Evaluation System for Computer Education

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Examining the Effects of a Real-Time, Knowledge-Aware Tool for Academic Writing Assessment

Skills Evaluation of Junior-Level Software Talents

Automating Transfer Credit Assessment in Student Mobility -- A Natural Language Processing-based Approach

A study of basketball skills assessment system

Automated Evaluation for Performance Test Scripts

The Design and Implementation of Test System for Expression Ability Evaluation

Automatic modelling of networked innovation outsourcing-oriented talent competency in the era of artificial intelligence

Identifying Talented Software Engineering Students through Data-driven Skill Assessment

Task Based Automatic Examination System for Sequenced Test

Learning Skill Equivalencies Across Platform Taxonomies

Computer-based assessment of collaborative problem solving skills: A systematic review of empirical research

Automatic grading system for C programming examination oriented to assessment of comprehensive practical abilities