Usability Testing for Smart IV Pumps Through Simulation-based Evaluation
Jiayin Chen,Kaifeng Liu,Foon-yee Chan,Zhenzhen Xie,Philip Lam,Yat-wo Liu,Hing-yu So,Calvin Ka-lun Or
DOI: https://doi.org/10.1177/1071181319631187
2019-11-01
Proceedings of the Human Factors and Ergonomics Society Annual Meeting
Abstract:Poorly designed user interfaces limit the effective use of medical devices, leading to user errors. Hence, before such devices are implemented, it is essential to identify potential usability issues. A common approach to testing usability is a simulation-based test, which allows practitioners to identify usability issues by observing how users interact with the product being tested. Infusion pumps are frequently used in medical settings, and much effort has been made to advance their design and features. To enhance patient safety, smart intravenous (IV) pumps with safety functions (e.g., drug library, alarms) have been developed. However, they do not always have usable interface designs. A simulation-based test was thus conducted in this study to examine the usability of smart IV pumps. Four smart IV pumps (Pumps 1, 2, 3, and 4) were tested. Table 1 presents their key characteristics. Thirty registered nurses with little experience of using such pumps were recruited from five units at three local public hospitals to participate in the test. They were asked to perform a series of tasks that simulated medication administration involving infusion pumps. A senior registered nurse and human factors professionals developed the following tasks for the test: priming an infusion set; loading an infusion set; accessing the drug library and selecting drugs; and programming a basic infusion. The tasks were performed in a simulated ward to reproduce the nurses’ actual practice. All tasks were performed with each pump, and the order in which the pumps were used was counterbalanced. The nurse participants wore video glasses that recorded their interactions with the pumps during the test. The study’s outcome measures included total task completion time, the number of deviations, the number of alarms, and the number of requests for help. Total task completion time was the amount of time taken to complete all of the tasks. A deviation occurred when the operation of a pump diverged from its standard operation, defined according to the instructions in the relevant pump user manual. Alarms were recorded when aural or visual alerts were caused by deviations and incorrect operation. Requests for help occurred when the participants asked the research assistant how to solve problems they had encountered. Repeated measures analysis of variance was performed to compare the total task completion time between the four pumps. Deviations, alarms, and requests for help were compared by the total number of occurrences. Table 1. Main features and operation of the pumps. Pump 1 Pump 2 Pump 3 Pump 4 Infusion set Gravity feeding set. Gravity feeding set. Gravity feeding set. Gravity feeding set. Flow sensor and Internal flow sensor; no External flow sensor; must Internal flow sensor; no Internal flow sensor; no installation need to install manually. be installed manually. need to install manually. need to install manually. How to access Automatic access via a page Automatic access via a page Users need to find and enter Automatic access via a “drug library” offering entry to the drug offering entry to the drug the main menu, then access page offering entry to the feature library. library. the drug library. drug library. How to select drugs in the drug library Press keys on the keypad. Press keys on the keypad. Press keys on the keypad or turn a dial. Press keys on the keypad. How to input numbers Use numeric keys and decimal keys. Use increment/decrement keys to increase or decrease value. Turn a dial to increase/decrease value. Use increment/decrement keys to increase or decrease value. The results of the test showed significant differences in total task completion time between the four pumps (F(3,87) = 14.56, p < 0.0001). The tasks were completed most quickly using Pump 1 (208.2 s), followed by Pump 2 (255.7 s), Pump 4 (304.6 s), and Pump 3 (369.2 s). The participants made the fewest deviations when using Pump 1 (36 deviations), followed by Pump 4 (92 deviations), Pump 2 (139 deviations), and Pump 3 (157 deviations). The participants triggered the fewest alarms when using Pump 1 (14 alarms), followed by Pump 2 (56 alarms), Pump 3 (67 alarms), and Pump 4 (67 alarms). As measured by their requests for help, the participants encountered the fewest problems with Pump 2 (8 requests for help), followed by Pump 1 (11 requests for help), Pump 4 (12 requests for help), and Pump 3 (27 requests for help). The overall findings were as follows. Pump 1 allowed a reasonable workflow, but the participants using this pump found it difficult to prime the infusion set and understand the alarm messages. Pump 2 was relatively easy to use, but the participants tended to forget to attach the external flow sensor. Using Pump 3, many participants found it difficult to access the drug library and program an infusion because the button labels were unclear. This may lead to misunderstandings and errors. In addition, using Pump 3 took the most time, which could have been due to the difficulty of inputting numbers using this pump. The design of Pump 3 requires users to input numbers by turning a dial, which takes longer than using keys. Using Pump 4, the participants were able to rapidly access the drug library and program a basic infusion, but some found it difficult to prime the infusion set. This study demonstrated that a simulation-based test is an effective way of identifying usability issues with medical devices. Developers of medical devices should improve their interface design to avoid ineffective use. Users should try to overcome usability issues by following the relevant user manuals closely. They should also receive sufficient training before using the devices. We observed a number of human errors during the test, and thus recommend that more training be provided in using the safety features of the pumps. Furthermore, practitioners should use different outcome measures to indicate performance when conducting similar kinds of simulation-based test, depending on the characteristics of the clinical settings in which the infusion pumps are used. Acknowledgments This study was conducted with the support of the Department of Industrial and Manufacturing Systems Engineering at the University of Hong Kong and the Theme-based Research Scheme of the Hong Kong Research Grants Council (grant number T32-102/14-N).