Abstract:Background: Eye movement tests remain significantly underutilized in emergency departments and primary healthcare units, despite their superior diagnostic sensitivity compared to neuroimaging modalities for the differential diagnosis of acute vertigo. This underutilization may be attributed to a potential lack of awareness regarding these tests and the absence of appropriate tools for detecting nystagmus. This study aimed to develop a nystagmus measurement algorithm using a lightweight deep-learning model that recognizes the ocular regions. Method: The deep learning model was used to segment the eye regions, detect blinking, and determine the pupil center. The model was trained using images extracted from video clips of a clinical battery of eye movement tests and synthesized images reproducing real eye movement scenarios using virtual reality. Each eye image was annotated with segmentation masks of the sclera, iris, and pupil, with gaze vectors of the pupil center for eye tracking. We conducted a comprehensive evaluation of model performance and its execution speeds in comparison to various alternative models using metrics that are suitable for the tasks. Results: The mean Intersection over Union values of the segmentation model ranged from 0.90 to 0.97 for different classes (sclera, iris, and pupil) across types of images (synthetic vs. real-world images). Additionally, the mean absolute error for eye tracking was 0.595 for real-world data and the F1 score for blink detection was ≥ 0.95, which indicates our model is performing at a very high level of accuracy. Execution speed was also the most rapid for ocular object segmentation under the same hardware condition as compared to alternative models. The prediction for horizontal and vertical nystagmus in real eye movement video revealed high accuracy with a strong correlation between the observed and predicted values ( r = 0.9949 for horizontal and r = 0.9950 for vertical; both p < 0.05). Conclusion: The potential of our model, which can automatically segment ocular regions and track nystagmus in real time from eye movement videos, holds significant promise for emergency settings or remote intervention within the field of neurotology.

Real-time Pupil Tracking from Monocular Video for Digital Puppetry

A Single-Camera Gaze Tracking System under Natural Light

Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction

Robust Real-Time Multi-User Pupil Detection and Tracking under Various Illumination and Large-Scale Head Motion

Persona: Real-Time Neural 3D Face Reconstruction for Visual Effects on Mobile Devices

PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection

Event Camera-Based Pupil Localization: Facilitating Training With Event-Style Translation of RGB Faces

Displaced Dynamic Expression Regression for Real-Time Facial Tracking and Animation

Real-time Facial Animation on Mobile Devices

PupilNet: Convolutional Neural Networks for Robust Pupil Detection

Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone

Implementation of a High-Accuracy Neural Network-Based Pupil Detection System for Real-Time and Real-World Applications

LiveCap: Real-time Human Performance Capture from Monocular Video

Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks

D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video

Real-Time Facial Segmentation and Performance Capture from RGB Input

Real-Time Gaze Tracking with Event-Driven Eye Segmentation

Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation

A Framework for Pupil Tracking with Event Cameras

Attention Mesh: High-fidelity Face Mesh Prediction in Real-time

Gazing into the Abyss: Real-time Gaze Estimation