Abstract:Eye gaze interfaces are an emerging technology that allows users to control graphical user interfaces (GUIs) simply by looking at them. However, using gaze-controlled GUIs can be a demanding task, resulting in high cognitive and physical load and fatigue. To address these challenges, we propose the concept and model of an adaptive human-assistive human–computer interface (HA-HCI) based on biofeedback. This model enables effective and sustainable use of computer GUIs controlled by physiological signals such as gaze data. The proposed model allows for analytical human performance monitoring and evaluation during human–computer interaction processes based on the damped harmonic oscillator (DHO) model. To test the validity of this model, the authors acquired gaze-tracking data from 12 healthy volunteers playing a gaze-controlled computer game and analyzed it using odd–even statistical analysis. The experimental findings show that the proposed model effectively describes and explains gaze-tracking performance dynamics, including subject variability in performance of GUI control tasks, long-term fatigue, and training effects, as well as short-term recovery of user performance during gaze-tracking-based control tasks. We also analyze the existing HCI and human performance models and develop an extension to the existing physiological models that allows for the development of adaptive user-performance-aware interfaces. The proposed HA-HCI model describes the interaction between a human and a physiological computing system (PCS) from the user performance perspective, incorporating a performance evaluation procedure that interacts with the standard UI components of the PCS and describes how the system should react to loss of productivity (performance). We further demonstrate the applicability of the HA-HCI model by designing an eye-controlled game. We also develop an analytical user performance model based on damped harmonic oscillation that is suitable for describing variability in performance of a PC game based on gaze tracking. The model's validity is tested using odd–even analysis, which demonstrates strong positive correlation. Individual characteristics of users established by the damped oscillation model can be used for categorization of players under their playing skills and abilities. The experimental findings suggest that players can be categorized as learners, whose damping factor is negative, and fatiguers, whose damping factor is positive. We find a strong positive correlation between amplitude and damping factor, indicating that good starters usually have higher fatigue rates, but slow starters have less fatigue and may even improve their performance during play. The proposed HA-HCI model and analytical user performance models provide a framework for developing an adaptive human-oriented HCI that enables monitoring, analysis, and increased performance of users working with physiological-computing-based user interfaces. The proposed models have potential applications in improving the usability of future human-assistive gaze-controlled interface systems.

Extension of HAAS for the Management of Cognitive Load

An adaptive human sensor framework for human–robot collaboration

An Online Framework for Cognitive Load Assessment in Assembly Tasks

Determining Cognitive Workload Using Physiological Measurements: Pupillometry and Heart-Rate Variability

Tools and methods for Human-Autonomy Teaming: Contributions to cognitive state monitoring and system adaptation

Human Digital Twin in the Context of Industry 5.0

A Forecasting Framework for Predicting Perceived Fatigue: Using Time Series Methods to Forecast Ratings of Perceived Exertion with Features from Wearable Sensors

Workload-Aware Systems and Interfaces for Cognitive Augmentation

Mental workload assessment by monitoring brain, heart, and eye with six biomedical modalities during six cognitive tasks

Cognitive Context Detection for Adaptive Automation

A human digital twin approach for fatigue-aware task planning in human-robot collaborative assembly

Fatigue evaluation in maintenance and assembly operations by digital human simulation

Digital ergonomic assessment to enhance the physical resilience of human-centric manufacturing systems in Industry 5.0

Real-time assessment of mental workload using psychophysiological measures and artificial neural networks

Human Digital Twin in Industry 5.0: A Holistic Approach to Worker Safety and Well-Being through Advanced AI and Emotional Analytics

A Human-Adaptive Model for User Performance and Fatigue Evaluation during Gaze-Tracking Tasks

A Virtual Reality Simulation Pipeline for Online Mental Workload Modeling

Pick the Right Co-Worker: Online Assessment of Cognitive Ergonomics in Human-Robot Collaborative Assembly

Fatigue evaluation in maintenance and assembly operations by digital human simulation in virtual environment

Real-Time Adaptive Industrial Robots: Improving Safety And Comfort In Human-Robot Collaboration

A Multi-disciplinary Assessments Tool for Human-Machine Interaction