Real-time virtual mouse system using RGB-D images and fingertip detection
Dinh-Son Tran,Ngoc-Huynh Ho,Hyung-Jeong Yang,Soo-Hyung Kim,Guee Sang Lee
DOI: https://doi.org/10.1007/s11042-020-10156-5
IF: 2.577
2020-11-23
Multimedia Tools and Applications
Abstract:Abstract A real-time fingertip-gesture-based interface is still challenging for human–computer interactions, due to sensor noise, changing light levels, and the complexity of tracking a fingertip across a variety of subjects. Using fingertip tracking as a virtual mouse is a popular method of interacting with computers without a mouse device. In this work, we propose a novel virtual-mouse method using RGB-D images and fingertip detection. The hand region of interest and the center of the palm are first extracted using in-depth skeleton-joint information images from a Microsoft Kinect Sensor version 2, and then converted into a binary image. Then, the contours of the hands are extracted and described by a border-tracing algorithm. The K-cosine algorithm is used to detect the fingertip location, based on the hand-contour coordinates. Finally, the fingertip location is mapped to RGB images to control the mouse cursor based on a virtual screen. The system tracks fingertips in real-time at 30 FPS on a desktop computer using a single CPU and Kinect V2. The experimental results showed a high accuracy level; the system can work well in real-world environments with a single CPU. This fingertip-gesture-based interface allows humans to easily interact with computers by hand.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering