Distributed Gesture Controlled Systems for Human-Machine Interface

Hans Johnson,Jafar Saniie

DOI: https://doi.org/10.1109/eIT53891.2022.9813764

2023-04-13

Abstract:This paper presents the design flow of an IoT human machine touchless interface. The device uses embedded computing in conjunction with the Leap Motion Controller to provide an accurate and intuitive touchless interface. Its main function is to augment current touchscreen devices in public spaces through a combination of computer vision technology, event driven programming, and machine learning. Especially following the COVID 19 pandemic, this technology is important for hygiene and sanitation purposes for public devices such as airports, food, and ATM kiosks where hundreds or even thousands of people may touch these devices in a single day. A prototype of the touchless interface was designed with a Leap Motion Controller housed on a Windows PC exchanging information with a Raspberry Pi microcontroller via internet connection.

Image and Video Processing,Human-Computer Interaction,Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to design a human - machine touch - control interface system based on the Internet of Things (IoT), aiming to provide an accurate and intuitive contact - free interface by combining embedded computing with the Leap Motion controller. Its main function is to enhance the functions of existing touch - screen devices in public places, especially by achieving this goal through a combination of computer vision technology, event - driven programming and machine learning. Especially after the COVID - 19 pandemic, this technology is particularly important for the hygiene and cleanliness of public devices (such as airports, food and ATM self - service terminals), because these devices may be touched by hundreds or even thousands of people every day.

The main problems mentioned in the paper include:
1. **Limitations of existing technologies**: Modern touch - interface technologies have performance deficiencies, resulting in these technologies not being widely integrated into practical applications.
2. **Hygiene problems**: During public health crises (such as COVID - 19), traditional touch - screen devices become risk points for virus transmission, and contact - free solutions are required to reduce the possibility of cross - infection.
3. **User experience**: Improve the accuracy and response speed of contact - free interfaces to provide a better user experience.

To solve these problems, the author proposes the design of a prototype system that uses the Leap Motion controller, Raspberry Pi microcontroller, camera and display to exchange information through Internet connections. The purpose of the system is to achieve precise control of the computer cursor through gesture control and send control commands to client devices through the network, thereby achieving contact - free interaction on public devices.