Demonstrating Eye Movement Biometrics in Virtual Reality

Dillon Lohr,Saide Johnson,Samantha Aziz,Oleg Komogortsev

DOI: https://doi.org/10.48550/arXiv.2207.02325

2022-07-06

Abstract:Thanks to the eye-tracking sensors that are embedded in emerging consumer devices like the Vive Pro Eye, we demonstrate that it is feasible to deliver user authentication via eye movement biometrics.

Human-Computer Interaction

What problem does this paper attempt to address?

An implementation of eye movement-driven biometrics in virtual reality

Dillon Lohr,Samuel-Hunter Berndt,Oleg Komogortsev

DOI: https://doi.org/10.1145/3204493.3208333

2018-06-14

Abstract:As eye tracking can reduce the computational burden of virtual reality devices through a technique known as foveated rendering, we believe not only that eye tracking will be implemented in all virtual reality devices, but that eye tracking biometrics will become the standard method of authentication in virtual reality. Thus, we have created a real-time eye movement-driven authentication system for virtual reality devices. In this work, we describe the architecture of the system and provide a specific implementation that is done using the FOVE head-mounted display. We end with an exploration into future topics of research to spur thought and discussion.
Evaluating Eye Movement Biometrics in Virtual Reality: A Comparative Analysis of VR Headset and High-End Eye-Tracker Collected Dataset

Mehedi Hasan Raju,Dillon J Lohr,Oleg V Komogortsev

2024-05-06

Abstract:Previous studies have shown that eye movement data recorded at 1000 Hz can be used to authenticate individuals. This study explores the effectiveness of eye movement-based biometrics (EMB) by utilizing data from an eye-tracking (ET)-enabled virtual reality (VR) headset (GazeBaseVR) and compares it to the performance using data from a high-end eye tracker (GazeBase) that has been downsampled to 250 Hz. The research also aims to assess the biometric potential of both binocular and monocular eye movement data. GazeBaseVR dataset achieves an equal error rate (EER) of 1.67% and a false rejection rate (FRR) at 10^-4 false acceptance rate (FAR) of 22.73% in a binocular configuration. This study underscores the biometric viability of data obtained from eye-tracking-enabled VR headset.

Human-Computer Interaction
2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

Shuangqing Wu,Yin Zhang,Sanyuan Zhang,Xiuzi Ye,Yiyu Cai,Jianmin Zheng,Soumita Ghosh,Wenyu Chen,Xiaozheng Jane Zhang

DOI: https://doi.org/10.1145/1900179.1900245

2010-01-01

Abstract:In this paper, a 2D motion detection bounded hand 3D trajectory tracking and gesture recognition system is proposed for virtual reality interactions. First, the Bayes decision rule for classification of background and foreground is utilized to automatically locate the hand that bounded within a rectangle, and then the trajectory of the hand in 3D space is tracked by mean shift particle filter and stereo imaging. The skin color feature is exploited for image matting that effectively segment the hand contour in video sequence automatically. Finally the hand gesture is recognized by the connected component analysis and line approximation. The proposed technique works without any markers or constraints, overcomes the disturbance of arms and faces in the scene, and can recognize multiple hands with different gestures under complex background.
The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

Brendan John,Sanjeev Koppal,Eakta Jain,Sophie Jorg

DOI: https://doi.org/10.1109/tvcg.2020.2973052

IF: 5.2

2020-05-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:The gaze behavior of virtual avatars is critical to social presence and perceived eye contact during social interactions in Virtual Reality. Virtual Reality headsets are being designed with integrated eye tracking to enable compelling virtual social interactions. This paper shows that the near infra-red cameras used in eye tracking capture eye images that contain iris patterns of the user. Because iris patterns are a gold standard biometric, the current technology places the user's biometric identity at risk. Our first contribution is an optical defocus based hardware solution to remove the iris biometric from the stream of eye tracking images. We characterize the performance of this solution with different internal parameters. Our second contribution is a psychophysical experiment with a same-different task that investigates the sensitivity of users to a virtual avatar's eye movements when this solution is applied. By deriving detection threshold values, our findings provide a range of defocus parameters where the change in eye movements would go unnoticed in a conversational setting. Our third contribution is a perceptual study to determine the impact of defocus parameters on the perceived eye contact, attentiveness, naturalness, and truthfulness of the avatar. Thus, if a user wishes to protect their iris biometric, our approach provides a solution that balances biometric protection while preventing their conversation partner from perceiving a difference in the user's virtual avatar. This work is the first to develop secure eye tracking configurations for VR/AR/XR applications and motivates future work in the area.

computer science, software engineering
Exploiting the Uncoordinated Privacy Protections of Eye Tracking and VR Motion Data for Unauthorized User Identification

Samantha Aziz,Oleg Komogortsev

2024-11-18

Abstract:Virtual reality (VR) devices use a variety of sensors to capture a rich body of user-generated data, which can be misused by malicious parties to covertly infer information about the user. Privacy-enhancing techniques seek to reduce the amount of personally identifying information in sensor data, but these techniques are typically developed for a subset of data streams that are available on the platform, without consideration for the auxiliary information that may be readily available from other sensors. In this paper, we evaluate whether body motion data can be used to circumvent the privacy protections applied to eye tracking data to enable user identification on a VR platform, and vice versa. We empirically show that eye tracking, headset tracking, and hand tracking data are not only informative for inferring user identity on their own, but contain complementary information that can increase the rate of successful user identification. Most importantly, we demonstrate that applying privacy protections to only a subset of the data available in VR can create an opportunity for an adversary to bypass those privacy protections by using other unprotected data streams that are available on the platform, performing a user identification attack as accurately as though a privacy mechanism was never applied. These results highlight a new privacy consideration at the intersection between eye tracking and VR, and emphasizes the need for privacy-enhancing techniques that address multiple technologies comprehensively.

Human-Computer Interaction
Beyond Gaze Points: Augmenting Eye Movement with Brainwave Data for Multimodal User Authentication in Extended Reality

Matin Fallahi,Patricia Arias-Cabarcos,Thorsten Strufe

2024-04-29

Abstract:The increasing adoption of Extended Reality (XR) in various applications underscores the need for secure and user-friendly authentication methods. However, existing methods can disrupt the immersive experience in XR settings, or suffer from higher false acceptance rates. In this paper, we introduce a multimodal biometric authentication system that combines eye movement and brainwave patterns, as captured by consumer-grade low-fidelity sensors. Our multimodal authentication exploits the non-invasive and hands-free properties of eye movement and brainwaves to provide a seamless XR user experience and enhanced security as well. Using synchronized eye and brainwave data collected from 30 participants through consumer-grade devices, we investigated whether twin neural networks can utilize these biometrics for identity verification. Our multimodal authentication system yields an excellent Equal Error Rate (EER) of 0.298\%, which means an 83.6\% reduction in EER compared to the single eye movement modality or a 93.9\% reduction in EER compared to the single brainwave modality.

Cryptography and Security
Development of a user evaluation system in virtual reality based on eye-tracking technology

DOI: https://doi.org/10.1007/s11042-023-14583-y

IF: 2.577

2023-02-19

Multimedia Tools and Applications

Abstract:This study proposes a user evaluation system based on a game engine, in which a user can measure their area or object of interest using an eye-tracking device. Eye tracking technology, which is widely used in various sectors such as human–computer interaction, user experience, and marketing research, measures the intentions of a human gazing on a part of their interest. However, creating a real-world test environment and model for user evaluation requires considerable time and money. Therefore, studies on user evaluation tests using virtual reality (VR) are being actively conducted. The VR user evaluation system based on the eye-tracking device proposed in this study was studied and developed using the unity game engine and an HTC VIVE Pro eye, in which an eye-tracking device was built in the head-mounted display. To make it appropriate for a VR environment, an eye-tracking algorithm was developed, and object- and surface-based eye-tracking and visualization technologies were applied. Moreover, the algorithm was developed to support component-based programming, so that an investigator can easily establish a test environment.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain–Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate

Jisoo Ha,Seonghun Park,Chang-Hwan Im

DOI: https://doi.org/10.3389/fninf.2022.758537

2022-02-24

Frontiers in Neuroinformatics

Abstract:Brain–computer interfaces (BCIs) based on electroencephalogram (EEG) have recently attracted increasing attention in virtual reality (VR) applications as a promising tool for controlling virtual objects or generating commands in a “hands-free” manner. Video-oculography (VOG) has been frequently used as a tool to improve BCI performance by identifying the gaze location on the screen, however, current VOG devices are generally too expensive to be embedded in practical low-cost VR head-mounted display (HMD) systems. In this study, we proposed a novel calibration-free hybrid BCI system combining steady-state visual-evoked potential (SSVEP)-based BCI and electrooculogram (EOG)-based eye tracking to increase the information transfer rate (ITR) of a nine-target SSVEP-based BCI in VR environment. Experiments were repeated on three different frequency configurations of pattern-reversal checkerboard stimuli arranged in a 3 × 3 matrix. When a user was staring at one of the nine visual stimuli, the column containing the target stimulus was first identified based on the user’s horizontal eye movement direction (left, middle, or right) classified using horizontal EOG recorded from a pair of electrodes that can be readily incorporated with any existing VR-HMD systems. Note that the EOG can be recorded using the same amplifier for recording SSVEP, unlike the VOG system. Then, the target visual stimulus was identified among the three visual stimuli vertically arranged in the selected column using the extension of multivariate synchronization index (EMSI) algorithm, one of the widely used SSVEP detection algorithms. In our experiments with 20 participants wearing a commercial VR-HMD system, it was shown that both the accuracy and ITR of the proposed hybrid BCI were significantly increased compared to those of the traditional SSVEP-based BCI in VR environment.

neurosciences,mathematical & computational biology
Assessing Saccadic Eye Movements With Head-Mounted Display Virtual Reality Technology

Yu Imaoka,Andri Flury,Eling D de Bruin

DOI: https://doi.org/10.3389/fpsyt.2020.572938

2020-09-17

Abstract:As our society is ageing globally, neurodegenerative disorders are becoming a relevant issue. Assessment of saccadic eye movement could provide objective values to help to understand the symptoms of disorders. HTC Corporation launched a new virtual reality (VR) headset, VIVE Pro Eye, implementing an infrared-based eye tracking technique together with VR technology. The purpose of this study is to evaluate whether the device can be used as an assessment tool of saccadic eye movement and to investigate the technical features of eye tracking. We developed a measurement system of saccadic eye movement with a simple VR environment on Unity VR design platform, following an internationally proposed standard saccade measurement protocol. We then measured the saccadic eye movement of seven healthy young adults to analyze the oculo-metrics of latency, peak velocity, and error rate of pro- and anti-saccade tasks: 120 trials in each task. We calculated these parameters based on the saccade detection algorithm that we have developed following previous studies. Consequently, our results revealed latency of 220.40 ± 43.16 ms, peak velocity of 357.90 ± 111.99°/s, and error rate of 0.24 ± 0.41% for the pro-saccade task, and latency of 343.35 ± 76.42 ms, peak velocity of 318.79 ± 116.69°/s, and error rate of 0.66 ± 0.76% for the anti-saccade task. In addition, we observed pupil diameter of 4.30 ± 1.15 mm (left eye) and 4.29 ± 1.08 mm (right eye) for the pro-saccade task, and of 4.21 ± 1.04 mm (left eye) and 4.22 ± 0.97 mm (right eye) for the anti-saccade task. Comparing between the descriptive statistics of previous studies and our results suggests that VIVE Pro Eye can function as an assessment tool of saccadic eye movement since our results are in the range of or close to the results of previous studies. Nonetheless, we found technical limitations especially about time-related measurement parameters. Further improvements in software and hardware of the device and measurement protocol, and more measurements with diverse age-groups and people with different health conditions are warranted to enhance the whole assessment system of saccadic eye movement.
A method for synchronized use of EEG and eye tracking in fully immersive VR

Olav F. P. Larsen,William G. Tresselt,Emanuel A. Lorenz,Tomas Holt,Grethe Sandstrak,Tor I. Hansen,Xiaomeng Su,Alexander Holt

DOI: https://doi.org/10.3389/fnhum.2024.1347974

IF: 3.473

2024-02-27

Frontiers in Human Neuroscience

Abstract:This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

psychology,neurosciences
3D Face Reconstruction and Gaze Tracking in the HMD for Virtual Interaction `

Shu-Yu Chen,Yu-Kun Lai,Shihong Xia,Paul Rosin,Lin Gao

DOI: https://doi.org/10.1109/tmm.2022.3156820

IF: 7.3

2022-01-01

IEEE Transactions on Multimedia

Abstract:With the rapid development of virtual reality (VR) technology, VR headsets, a.k.a. Head-Mounted Displays (HMDs), are widely available, allowing immersive 3D content to be viewed. A natural need for truly immersive VR is to allow bidirectional communication: the user should be able to interact with the virtual world using facial expressions and eye gaze, in addition to traditional means of interaction. The typical application scenario includes VR virtual conferencing and virtual roaming, where ideally users are able to see other users expressions and have eye contact with them in the virtual world. In addition, eye gaze also provides a natural means of interaction with virtual objects. Despite significant achievements in recent years for reconstruction of 3D faces from RGB or RGB-D images, it remains a challenge to reliably capture and reconstruct 3D facial expressions including eye gaze when the user is wearing an HMD, because the majority of the face is occluded, especially those areas around the eyes which are essential for recognizing facial expressions and eye gaze. In this paper, we introduce a novel real-time system that is able to capture and reconstruct 3D faces wearing HMDs, and robustly recover eye gaze. We further propose a novel method to map eye gaze directions to the 3D virtual world, which provides a novel and useful interactive mode in VR. We compare our method with state of-the-art techniques both qualitatively and quantitatively, and demonstrate the effectiveness of our system using live capture.

computer science, information systems,telecommunications, software engineering
Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges

Efe Bozkir,Süleyman Özdel,Mengdi Wang,Brendan David-John,Hong Gao,Kevin Butler,Eakta Jain,Enkelejda Kasneci

DOI: https://doi.org/10.48550/arXiv.2305.14080

IF: 6.4588

2023-05-23

Human-Computer Interaction

Abstract:Latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life. Eye tracking offers not only a hands-free way of interaction but also the possibility of a deeper understanding of human visual attention and cognitive processes in VR. Despite these possibilities, eye-tracking data also reveal privacy-sensitive attributes of users when it is combined with the information about the presented stimulus. To address these possibilities and potential privacy issues, in this survey, we first cover major works in eye tracking, VR, and privacy areas between the years 2012 and 2022. While eye tracking in the VR part covers the complete pipeline of eye-tracking methodology from pupil detection and gaze estimation to offline use and analyses, as for privacy and security, we focus on eye-based authentication as well as computational methods to preserve the privacy of individuals and their eye-tracking data in VR. Later, taking all into consideration, we draw three main directions for the research community by mainly focusing on privacy challenges. In summary, this survey provides an extensive literature review of the utmost possibilities with eye tracking in VR and the privacy implications of those possibilities.
Towards Everyday Virtual Reality through Eye Tracking

Efe Bozkir

DOI: https://doi.org/10.48550/arXiv.2203.15703

IF: 6.4588

2022-03-29

Human-Computer Interaction

Abstract:With developments in computer graphics, hardware technology, perception engineering, and human-computer interaction, virtual reality and virtual environments are becoming more integrated into our daily lives. Head-mounted displays, however, are still not used as frequently as other mobile devices such as smart phones and watches. With increased usage of this technology and the acclimation of humans to virtual application scenarios, it is possible that in the near future an everyday virtual reality paradigm will be realized. When considering the marriage of everyday virtual reality and head-mounted displays, eye tracking is an emerging technology that helps to assess human behaviors in a real time and non-intrusive way. Still, multiple aspects need to be researched before these technologies become widely available in daily life. Firstly, attention and cognition models in everyday scenarios should be thoroughly understood. Secondly, as eyes are related to visual biometrics, privacy preserving methodologies are necessary. Lastly, instead of studies or applications utilizing limited human participants with relatively homogeneous characteristics, protocols and use-cases for making such technology more accessible should be essential. In this work, taking the aforementioned points into account, a significant scientific push towards everyday virtual reality has been completed with three main research contributions.
Facial Motion Capture System Based on Facial Electromyogram and Electrooculogram for Immersive Social Virtual Reality Applications

Chunghwan Kim,Ho-Seung Cha,Junghwan Kim,HwyKuen Kwak,WooJin Lee,Chang-Hwan Im

DOI: https://doi.org/10.3390/s23073580

2023-03-29

Abstract:With the rapid development of virtual reality (VR) technology and the market growth of social network services (SNS), VR-based SNS have been actively developed, in which 3D avatars interact with each other on behalf of the users. To provide the users with more immersive experiences in a metaverse, facial recognition technologies that can reproduce the user's facial gestures on their personal avatar are required. However, it is generally difficult to employ traditional camera-based facial tracking technology to recognize the facial expressions of VR users because a large portion of the user's face is occluded by a VR head-mounted display (HMD). To address this issue, attempts have been made to recognize users' facial expressions based on facial electromyogram (fEMG) recorded around the eyes. fEMG-based facial expression recognition (FER) technology requires only tiny electrodes that can be readily embedded in the HMD pad that is in contact with the user's facial skin. Additionally, electrodes recording fEMG signals can simultaneously acquire electrooculogram (EOG) signals, which can be used to track the user's eyeball movements and detect eye blinks. In this study, we implemented an fEMG- and EOG-based FER system using ten electrodes arranged around the eyes, assuming a commercial VR HMD device. Our FER system could continuously capture various facial motions, including five different lip motions and two different eyebrow motions, from fEMG signals. Unlike previous fEMG-based FER systems that simply classified discrete expressions, with the proposed FER system, natural facial expressions could be continuously projected on the 3D avatar face using machine-learning-based regression with a new concept named the virtual blend shape weight, making it unnecessary to simultaneously record fEMG and camera images for each user. An EOG-based eye tracking system was also implemented for the detection of eye blinks and eye gaze directions using the same electrodes. These two technologies were simultaneously employed to implement a real-time facial motion capture system, which could successfully replicate the user's facial expressions on a realistic avatar face in real time. To the best of our knowledge, the concurrent use of fEMG and EOG for facial motion capture has not been reported before.
EyeVeri: A secure and usable approach for smartphone user authentication

Chen Song,Aosen Wang,Kui Ren,Wenyao Xu

DOI: https://doi.org/10.1109/INFOCOM.2016.7524367

2016-01-01

Abstract:As mobile technology grows rapidly, the smartphone has become indispensable for transmitting private user data, storing the sensitive corporate files, and conducting secure payment transactions. However, with mobile security research lagging, smartphones are extremely vulnerable to unauthenticated access. In this paper, we present, EyeVeri, a novel eye-movement-based authentication system for smartphone security protection. Specifically, EyeVeri tracks human eye movement through the built-in front camera and applies the signal processing and pattern matching techniques to explore volitional and non-volitional gaze patterns for access authentication. Through a comprehensive user study, EyeVeri performs well and is a promising approach for smartphone user authentication. We also discuss the evaluation results in-depth and analyze opportunities for future work.
Establishing a Baseline for Gaze-driven Authentication Performance in VR: A Breadth-First Investigation on a Very Large Dataset

Dillon Lohr,Michael J. Proulx,Oleg Komogortsev

2024-07-12

Abstract:This paper performs the crucial work of establishing a baseline for gaze-driven authentication performance to begin answering fundamental research questions using a very large dataset of gaze recordings from 9202 people with a level of eye tracking (ET) signal quality equivalent to modern consumer-facing virtual reality (VR) platforms. The size of the employed dataset is at least an order-of-magnitude larger than any other dataset from previous related work. Binocular estimates of the optical and visual axes of the eyes and a minimum duration for enrollment and verification are required for our model to achieve a false rejection rate (FRR) of below 3% at a false acceptance rate (FAR) of 1 in 50,000. In terms of identification accuracy which decreases with gallery size, we estimate that our model would fall below chance-level accuracy for gallery sizes of 148,000 or more. Our major findings indicate that gaze authentication can be as accurate as required by the FIDO standard when driven by a state-of-the-art machine learning architecture and a sufficiently large training dataset.

Computer Vision and Pattern Recognition,Human-Computer Interaction
Measuring eye-tracking accuracy and its impact on usability in apple vision pro

Zehao Huang,Gancheng Zhu,Xiaoting Duan,Rong Wang,Yongkai Li,Shuai Zhang,Zhiguo Wang

2024-08-14

Abstract:With built-in eye-tracking cameras, the recently released Apple Vision Pro (AVP) mixed reality (MR) headset features gaze-based interaction, eye image rendering on external screens, and iris recognition for device unlocking. One of the technological advancements of the AVP is its heavy reliance on gaze- and gesture-based interaction. However, limited information is available regarding the technological specifications of the eye-tracking capability of the AVP, and raw gaze data is inaccessible to developers. This study evaluates the eye-tracking accuracy of the AVP with two sets of tests spanning both MR and virtual reality (VR) applications. This study also examines how eye-tracking accuracy relates to user-reported usability. The results revealed an overall eye-tracking accuracy of 1.11° and 0.93° in two testing setups, within a field of view (FOV) of approximately 34° x 18°. The usability and learnability scores of the AVP, measured using the standard System Usability Scale (SUS), were 75.24 and 68.26, respectively. Importantly, no statistically reliable correlation was found between eye-tracking accuracy and usability scores. These results suggest that eye-tracking accuracy is critical for gaze-based interaction, but it is not the sole determinant of user experience in VR/AR.

Human-Computer Interaction
eyemR-Vis: A Mixed Reality System to Visualise Bi-Directional Gaze Behavioural Cues Between Remote Collaborators

Allison Jing,Kieran William May,Mahnoor Naeem,Gun Lee,Mark Billinghurst

DOI: https://doi.org/10.1145/3411763.3451545

2021-05-08

Abstract:This demonstration shows eyemR-Vis, a 360 panoramic Mixed Reality collaboration system that translates gaze behavioural cues to bi-directional visualisations between a local host (AR) and a remote collaborator (VR). The system is designed to share dynamic gaze behavioural cues as bi-directional spatial virtual visualisations between a local host and a remote collaborator. This enables richer communication of gaze through four visualisation techniques: browse, focus, mutual-gaze, and fixated circle-map. Additionally, our system supports simple bi-directional avatar interaction as well as panoramic video zoom. This makes interaction in the normally constrained remote task space more flexible and relatively natural. By showing visual communication cues that are physically inaccessible in the remote task space through reallocating and visualising the existing ones, our system aims to provide a more engaging and effective remote collaboration experience.
A New Terrain in HCI: Emotion Recognition Interface using Biometric Data for an Immersive VR Experience

Jaehyun Nam,Hyesun Chung,Young ah Seong,Honggu Lee

DOI: https://doi.org/10.48550/arXiv.1912.01177

IF: 6.4588

2019-12-03

Human-Computer Interaction

Abstract:Emotion recognition technology is crucial in providing a personalized user experience. It is especially important in virtual reality(VR) to assess the user's emotions to enhance their sense of immersion. We propose an emotion recognition interface that incorporates the user's biometric data with machine learning technology for increasing user engagement in VR. Our key technologies include brainwave sensors and eye-tracking cameras embedded in a VR headset, which seamlessly acquire physiological signals, and secondly, an attractiveness recognition algorithm that uses bio-signals to predict the user's attraction on visual stimuli. We conducted experiments to test the performance of the system, and also interviewed experts and participants to acquire opinions on the system. This study demonstrated the technical feasibility of our system with high accuracy and usability. Interviewees expected that the interface will be actively used in the context of various applications. Our proposed interface could contribute to an immersive VR experience design.
Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality

Ulrik Günther,Kyle I. S. Harrington,Raimund Dachselt,Ivo F. Sbalzarini,Kyle I.S. Harrington

DOI: https://doi.org/10.48550/arXiv.2005.00387

IF: 6.4588

2020-05-01

Human-Computer Interaction

Abstract:We present Bionic Tracking, a novel method for solving biological cell tracking problems with eye tracking in virtual reality using commodity hardware. Using gaze data, and especially smooth pursuit eye movements, we are able to track cells in time series of 3D volumetric datasets. The problem of tracking cells is ubiquitous in developmental biology, where large volumetric microscopy datasets are acquired on a daily basis, often comprising hundreds or thousands of time points that span hours or days. The image data, however, is only a means to an end, and scientists are often interested in the reconstruction of cell trajectories and cell lineage trees. Reliably tracking cells in crowded three-dimensional space over many timepoints remains an open problem, and many current approaches rely on tedious manual annotation and curation. In our Bionic Tracking approach, we substitute the usual 2D point-and-click annotation to track cells with eye tracking in a virtual reality headset, where users simply have to follow a cell with their eyes in 3D space in order to track it. We detail the interaction design of our approach and explain the graph-based algorithm used to connect different time points, also taking occlusion and user distraction into account. We demonstrate our cell tracking method using the example of two different biological datasets. Finally, we report on a user study with seven cell tracking experts, demonstrating the benefits of our approach over manual point-and-click tracking.

Demonstrating Eye Movement Biometrics in Virtual Reality

An implementation of eye movement-driven biometrics in virtual reality

Evaluating Eye Movement Biometrics in Virtual Reality: A Comparative Analysis of VR Headset and High-End Eye-Tracker Collected Dataset

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

Exploiting the Uncoordinated Privacy Protections of Eye Tracking and VR Motion Data for Unauthorized User Identification

Beyond Gaze Points: Augmenting Eye Movement with Brainwave Data for Multimodal User Authentication in Extended Reality

Development of a user evaluation system in virtual reality based on eye-tracking technology

Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain–Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate

Assessing Saccadic Eye Movements With Head-Mounted Display Virtual Reality Technology

A method for synchronized use of EEG and eye tracking in fully immersive VR

3D Face Reconstruction and Gaze Tracking in the HMD for Virtual Interaction `

Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges

Towards Everyday Virtual Reality through Eye Tracking

Facial Motion Capture System Based on Facial Electromyogram and Electrooculogram for Immersive Social Virtual Reality Applications

EyeVeri: A secure and usable approach for smartphone user authentication

Establishing a Baseline for Gaze-driven Authentication Performance in VR: A Breadth-First Investigation on a Very Large Dataset

Measuring eye-tracking accuracy and its impact on usability in apple vision pro

eyemR-Vis: A Mixed Reality System to Visualise Bi-Directional Gaze Behavioural Cues Between Remote Collaborators

A New Terrain in HCI: Emotion Recognition Interface using Biometric Data for an Immersive VR Experience

Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality