Deciding to Stop Early or Continue the Experiment After Checking p-Values at Interim Points: Introducing Group Sequential Designs to UI-Based Comparative Studies

Shota Yamanaka
DOI: https://doi.org/10.1080/10447318.2024.2407662
IF: 4.92
2024-10-09
International Journal of Human-Computer Interaction
Abstract:Null hypothesis significance testing (NHST) is widely used in the field of human-computer interaction (HCI), and caution has been advised against deciding whether to stop or continue an experiment based on checking p -values. However, in clinical trials, NHST with interim analysis is commonly employed by adjusting the α levels to control Type I error rates. This approach, known as a group sequential design, can also be beneficial in the HCI field. We analyzed existing datasets of target-pointing tasks using group sequential designs and experimentally demonstrated that, depending on the α -level correction method, we could reduce the number of participants by 40.4–56.5%. Therefore, employing group sequential designs enables significant time and cost savings for both participants and researchers in the HCI field.
computer science, cybernetics,ergonomics
What problem does this paper attempt to address?