A Comparison of Several Subjective Rating Scales of Cognitive Load

Sun Chongyong,Liu Dianzhi

DOI: https://doi.org/10.16719/j.cnki.1671-6981.2013.01.014

2013-01-01

Abstract:After Cognitive Load Theory(CLT)was proposed by John Sweller in the 1980s,researchers began to invent new methods for measuring cognitive load.At present,the measurements can be divided into three categories;subjective measures,task performance measures and physiological measures.Because of its simplicity,practicability and convenience,subjective measures were the most popular in recent influential research on cognitive load.According to some statistics,the Paas Cognitive Load Scale(the PAAS)designed by Paas in 1993 was mostly used in the previous research.However,some researchers were critical of it,especially of the sensitivity and validity of PAAS,because the number of items was limited and it was easy to cause social desirability effects.Therefore,the objective of the research was to identify the most sensitive and effective subjective measurement tools by comparing three subjective rating scales, and to test the sensitivity of response time of the secondary-task which was used as a reference object for subjective measures. In the dual-task experiment paradigm,the primary task was face recognition,which was to choose the only right answer from four faces according to the criterion ordered by the researcher,and the secondary task was mental calculation,which was to judge whether a two-digit number can be divided with no remainder by three or not.We used a 3 x 4 mixed design,with the between-subjects factor being a category of the subjective rating scale and the within-subjects factor being task difficulty.The subjective rating scale included three categories,which were the PAAS,the Workload Profile Index Ratings(the WP) and the NASA Task Load Index(the TLX).The task difficulty included four categories,which were matched by the difficulty level of the primary task(easy or difficult) and the secondary task(easy or difficult).A sample of 60 college students volunteered to participate in the study.Ages ranged from 18 to 22 and all were right-handed.The subjects were randomly assigned to three groups of the same size;20 subjects filled out the PAAS,another 20 answered the WP,and the remaining 20 filled in the TLX.The experiment was carried out on the computer.The program was designed by E-prime psychology software so as to control the presentation time of experiment materials and measure the response time and accuracy of the secondary task. The hypotheses were testified on the whole.The results showed that,in the present task condition,both the sensitivity and intrusiveness of response time of the secondary-task were better and it could be used as a reference object for subjective measures.Meanwhile, the sensitivity and validity of both the WP and the PAAS were better,and were higher than those of the TLX.At the same time, the sensitivity and diagnosticity of the WP were higher than those of the PAAS.In conclusion,taking all evaluating indexes into consideration, the WP was the most effective and perfective subjective rating scale of the three,especially for the task of medium or lower difficulty.