Abstract:Across a plethora of social situations, we touch others in natural and intuitive ways to share thoughts and emotions, such as tapping to get one's attention or caressing to soothe one's anxiety. A deeper understanding of these human-to-human interactions will require, in part, the precise measurement of skin-to-skin physical contact. Among prior efforts, each measurement approach exhibits certain constraints, e.g., motion trackers do not capture the precise shape of skin surfaces, while pressure sensors impede skin-to-skin contact. In contrast, this work develops an interference-free 3D visual tracking system using a depth camera to measure the contact attributes between the bare hand of a toucher and the forearm of a receiver. The toucher's hand is tracked as a posed and positioned mesh by fitting a hand model to detected 3D hand joints, whereas a receiver's forearm is extracted as a 3D surface updated upon repeated skin contact. Based on a contact model involving point clouds, the spatiotemporal changes of hand-to-forearm contact are decomposed as six, high-resolution, time-series contact attributes, i.e., contact area, indentation depth, absolute velocity, and three orthogonal velocity components, together with contact duration. To examine the system's capabilities and limitations, two types of experiments were performed. First, to evaluate its ability to discern human touches, one person delivered cued social messages, e.g., happiness, anger, sympathy, to another person using their preferred gestures. The results indicated that messages and gestures, as well as the identities of the touchers, were readily discerned from their contact attributes. Second, the system's spatiotemporal accuracy was validated against measurements from independent devices, including an electromagnetic motion tracker, sensorized pressure mat, and laser displacement sensor. While validated here in the context of social communication, this system is extendable to human touch interactions such as maternal care of infants and massage therapy.

FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer

Using Smartwatches to Detect Face Touching

FaceTouch: Practical Face Touch Detection with a Multimodal Wearable System for Epidemiological Surveillance

CovidAlert -- A Wristwatch-based System to Alert Users from Face Touching

WristSonic: Enabling Fine-grained Hand-Face Interactions on Smartwatches Using Active Acoustic Sensing

Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera

Counting Finger and Wrist Movements Using Only a Wrist-Worn, Inertial Measurement Unit: Toward Practical Wearable Sensing for Hand-Related Healthcare Applications

Hand Gesture Recognition on a Resource-Limited Interactive Wristband

Ipand: Accurate Gesture Input with Ambient Acoustic Sensing on Hand.

Social Distancing Alert with Smartwatches

Waving Gesture Analysis for User Authentication in the Mobile Environment

You Can Wash Hands Better: Accurate Daily Handwashing Assessment with Smartwatches

Acoustic Sensing-based Hand Gesture Detection for Wearable Device Interaction

It's All in the Touch: Authenticating Users with HOST Gestures on Multi-Touch Screen Devices

Unstructured Handwashing Recognition Using Smartwatch to Reduce Contact Transmission of Pathogens

TapWristband: A Wearable Keypad System Based on Wrist Vibration Sensing

Comparative Study of Gesture Recognition Based on Accelerometer and Photoplethysmography Sensor for Gesture Interactions in Wearable Devices

3D Visual Tracking to Quantify Physical Contact Interactions in Human-to-Human Touch

ShadowTouch: Enabling Free-Form Touch-Based Hand-to-Surface Interaction with Wrist-Mounted Illuminant by Shadow Projection

Hygiea+: Toward Energy-Efficient and Highly Accurate Toothbrushing Monitoring via Wrist-Worn Gesture Sensing

Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-Accelerometers.