Calibration-Free Mobile Eye-Tracking Using Corneal Imaging

Moayad Mokatren,Tsvi Kuflik,Ilan Shimshoni

DOI: https://doi.org/10.3390/s24041237

IF: 3.9

2024-02-15

Sensors

Abstract:In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system's mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user's point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to implement a calibration - free eye - tracking system on mobile devices. Traditional eye - tracking systems usually require users to perform cumbersome calibration procedures, which not only affect the user experience but also make it difficult to maintain the accuracy of calibration in the mobile environment. This paper proposes an automatic calibration method based on corneal imaging. By using three cameras (an infrared eye camera, an RGB eye camera, and a front - scene RGB camera), a reliable corneal imaging system is constructed, thereby achieving continuous and reliable estimation of the user's fixation point.

### Main problems
1. **Reduce user burden**: The traditional calibration process requires users to operate according to specific instructions, which is not only time - consuming but may also inconvenience users. The goal of this paper is to develop a system that can be automatically calibrated without user intervention.
2. **Improve accuracy in the mobile environment**: In the mobile environment, the position and angle of the device may change, causing calibration to fail. This paper ensures that the system can maintain high accuracy when the user moves through automatic calibration technology.
3. **Achieve real - time eye - tracking**: This paper aims to develop a system that can calculate the user's fixation point in real - time so as to provide immediate feedback in various application scenarios.

### Solutions
- **Multi - camera system**: Use three cameras (infrared eye camera, RGB eye camera, and front - scene RGB camera) to capture the user's eye images and front - scene images.
- **Deep - learning algorithm**: Use deep - learning algorithms to detect and track the iris center and generate a personalized 3D eyeball model for the user.
- **3D geometric calculation**: Through 3D geometric calculation, convert the image - plane coordinates of the iris center to the image - plane of the RGB eye camera, and then further convert them to the image - plane of the front - scene camera.
- **Automatic calibration**: The system can complete calibration without any special operations by the user through automatic calibration technology.

### Experimental results
- **Average indoor error**: 1.6°
- **Average outdoor error**: 1.69°

These results indicate that the system performs very well in practical applications, and its accuracy is comparable to or even better than existing advanced methods.

### Key contributions
1. **Automatic calibration technology**: Automatic calibration without user intervention is achieved, improving the ease - of - use and accuracy of the system.
2. **Multi - camera cooperation**: Through the cooperation of three cameras, the limitations of single - camera systems in complex lighting conditions are solved.
3. **Real - time eye - tracking**: Real - time eye - tracking can be achieved on mobile devices, which is suitable for multiple application scenarios.

Through these innovations, this paper provides a new direction for the development of mobile eye - tracking technology and is expected to be widely used in human - computer interaction, augmented reality, and other fields in the future.