Abstract:This study presents efficient vision-based finger detection, tracking, and event identification techniques and a low-cost hardware framework for multi-touch sensing and display applications. The proposed approach uses a fast bright-blob segmentation process based on automatic multilevel histogram thresholding to extract the pixels of touch blobs obtained from scattered infrared lights captured by a video camera. The advantage of this automatic multilevel thresholding approach is its robustness and adaptability when dealing with various ambient lighting conditions and spurious infrared noises. To extract the connected components of these touch blobs, a connected-component analysis procedure is applied to the bright pixels acquired by the previous stage. After extracting the touch blobs from each of the captured image frames, a blob tracking and event recognition process analyzes the spatial and temporal information of these touch blobs from consecutive frames to determine the possible touch events and actions performed by users. This process also refines the detection results and corrects for errors and occlusions caused by noise and errors during the blob extraction process. The proposed blob tracking and touch event recognition process includes two phases. First, the phase of blob tracking associates the motion correspondence of blobs in succeeding frames by analyzing their spatial and temporal features. The touch event recognition process can identify meaningful touch events based on the motion information of touch blobs, such as finger moving, rotating, pressing, hovering, and clicking actions. Experimental results demonstrate that the proposed vision-based finger detection, tracking, and event identification system is feasible and effective for multi-touch sensing applications in various operational environments and conditions.

Touch Point Prediction for Interactive Public Displays Based on Camera Images

COMTIS: Customizable Touchless Interaction System for Large Screen Visualization

A touchable virtual screen interaction system with handheld Kinect camera

A Novel PRO-CAM Based Interactive Display Surface

Single-tap Latency Reduction with Single- or Double- tap Prediction

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

A new finger touch detection algorithm and prototype system architecture for pervasive bare-hand human computer interaction

V-Hands: Touchscreen-based Hand Tracking for Remote Whiteboard Interaction

A Study on Interaction Prediction for Reducing Interaction Latency in Remote Mixed Reality Collaboration

Bare-fingers Touch Detection by the Button's Distortion in a Projector–Camera System

TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric Vision

Self-supervised Spatio-Temporal Graph Mask-Passing Attention Network for Perceptual Importance Prediction of Multi-point Tactility

ShadowTouch: Enabling Free-Form Touch-Based Hand-to-Surface Interaction with Wrist-Mounted Illuminant by Shadow Projection

Vision-based finger detection, tracking, and event identification techniques for multi-touch sensing and display systems

Responsive-ExtendedHand: Adaptive Visuo-Haptic Feedback Recognizing Object Property With RGB-D Camera for Projected Extended Hand

SpeciFingers: Finger Identification and Error Correction on Capacitive Touchscreens

Full Screen Touch Detection for the Virtual Touch Screen

Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control

Hand Gesture Interface Based on Improved Adaptive Hand Area Detection and Contour Signature

Real-Time Hand Gesture Spotting and Recognition Using RGB-D Camera and 3D Convolutional Neural Network

Recognizing Unintentional Touch on Interactive Tabletop.