Contactless Human–computer Interaction System Based on Three-Dimensional Holographic Display and Gesture Recognition
Yixin Yang,Yunhui Gao,Kexuan Liu,Zehao He,Liangcai Cao
DOI: https://doi.org/10.1007/s00340-023-08128-2
2023-01-01
Applied Physics B
Abstract:Contactless interaction provides a hygienic, safe, and intelligent way for communication between human and computers. As computing systems continue to advance, there is a growing focus on input methods and keyboard interaction approaches. However, current research on virtual keyboard systems remains confined to two-dimensional interfaces. The existing contactless interaction implementations are generally limited by handheld tools, wearable devices, or projection screens. In this work, we propose a contactless three-dimensional (3D) virtual keyboard system, which combines gesture recognition and holographic display. A hand-tracking sensor is used to collect the user’s gestures and fingertip positions as input. A spatial light modulator is utilized to generate the corresponding holographic 3D display images. The hand-tracking sensor and display module are controlled in a synchronous manner by a personal computer. The robust and real-time system is demonstrated through user-interactive experiments with a 3D virtual keyboard. The scalability of the proposed human–computer interaction system supports a wide range of applications, including education, communication, disease prevention, information security, and virtual reality.