META: Multidimensional Evaluation of Testing Ability

Tianqi Zhou,Jiawei Liu,Yifan Wang,Zhenyu Chen
DOI: https://doi.org/10.1145/3510454.3516867
2022-01-01
Abstract:As the market's demand for software quality continues increasing, companies increase demand for excellent testing engineers. The online testing platform cultivates students by offering software testing courses. Students are encouraged to submit test codes by exams on the online testing platform during the course evaluation. However, the problem of how to effectively assess the test code written by students is to be solved. This paper implements a multidimensional evaluation system named META for software testing based on an online testing platform. META is designed to address the problem of how to evaluate testing effectiveness systematically. This paper evaluates students' testing effectiveness in seven dimensions for three software testing types: developer unit testing, web application testing, and mobile application testing, combining test codes and test behaviours. For the validity of META, 14 exams are selected from MOOCTest for an experiment in this paper, of which ten exams for developer unit testing, three exams for mobile application testing, and one exam for web application testing, involving 718 students participating in the exam and 26666 records submitted. The experimental results show that META can present significant variability in different dimensions for different students with similar scores. Video URL: https://www.youtube.com/watch?v=EiCSMtefPMU.
What problem does this paper attempt to address?