Abstract:Gesture recognition is an emerging technology of human–computer interaction. Compared to the conventional technologies such as wearable devices, cameras, and radars, ultrasound-based gesture recognition has the advantages of low cost, low power consumption, and appropriates to fog or dark environment. In this study, we propose a gesture trajectory recognition device via ultrasonic signal, which is successfully applied in recognition and classification for gestures in 3-D space. First, the sensing hardware and software configurations for transmitting and receiving ultrasonic signals are designed, which can accurately obtain ultrasonic echo. Effects of different structures on the ultrasound scattering are explored. An adaptive filter based on wavelet packet decomposition is utilized to remove the noise in the ultrasonic echo. Second, the vibration principle of the ultrasonic sensor is modeled, and the rising envelope of ultrasound is fit by the quadratic curve to accurately extract the time of flight (TOF). Then, the error analysis and compensation are performed. An active 3-D positioning model is constructed and the positioning and tracking algorithm of gesture motion is theoretically established. A binomial fitting algorithm-based sliding template is proposed. To process the abnormal value and locate the start position of the gesture, the data association criterion and the maximum displacement threshold are introduced. Projection, dimensionality reduction, and reconstruction are performed on the trajectory. Then, the distance feature and the direction feature are extracted, and the hidden Markov model (HMM) is applied to realize the classification and recognition of gesture trajectory. The experiment shows that the 3-D gesture recognition system based on ultrasound can recognize a total of 36 character gestures from 0 to 9 and A to Z, with a recognition rate of 90.53%.

Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic Signals

SmartSit: Sitting Posture Recognition Through Acoustic Sensing on Smartphones

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

In-Hand 3D Object Reconstruction from a Monocular RGB Video

Beyond-Voice: Towards Continuous 3D Hand Pose Tracking on Commercial Home Assistant Devices

Mmhand: 3D Hand Pose Estimation Leveraging Mmwave Signals

Real Time 3D-Handwritten Character and Gesture Recognition for Smartphone

HandOS: 3D Hand Reconstruction in One Stage

Poster - Sonicnect: Accurate Hands-Free Gesture Input System With Smart Acoustic Sensing

EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband

SVoice: Enabling Voice Communication in Silence Via Acoustic Sensing on Commodity Devices.

Deep Learning-Based 3D Pose Reconstruction of an Underwater Soft Robotic Hand and Its Biomimetic Evaluation

Ultrasound-Based 3-D Gesture Recognition: Signal Optimization, Trajectory, and Feature Classification

Biomimetic Evaluation of an Underwater Soft Hand Through Deep Learning-based 3D Pose Reconstruction

SVoice

Smartphone-based Handwritten Signature Verification Using Acoustic Signals.

WristSonic: Enabling Fine-grained Hand-Face Interactions on Smartwatches Using Active Acoustic Sensing

SonicOperator: Ultrasonic gesture recognition with deep neural network on mobiles

Ipand: Accurate Gesture Input with Ambient Acoustic Sensing on Hand.

HCI on the Table: Robust Gesture Recognition Using Acoustic Sensing in Your Hand

SignID: Acoustic-based Identification with Single Sign Gesture