Abstract:Recently, gesture recognition has gained considerable attention in emerging applications (e.g., AR/VR systems) to provide a better user experience for human-computer interaction. Existing solutions usually recognize the gestures based on wearable sensors or specialized signals (e.g., WiFi, acoustic and visible light), but they are either incurring high energy consumption or susceptible to the ambient environment, which prevents them from efficiently sensing the fine-grained finger movements. In this paper, we present RF-finger, a device-free system based on Commercial-Off-The-Shelf (COTS) RFID, which leverages a tag array on a letter-size paper to sense the fine-grained finger movements performed in front of the paper. Particularly, we focus on two kinds of sensing modes: finger tracking recovers the moving trace of finger writings; multi-touch gesture recognition identifies the multi-touch gestures involving multiple fingers. Specifically, we build a theoretical model to extract the fine-grained reflection feature from the raw RF -signal, which describes the finger influence on the tag array in cm- level resolution. For the finger tracking, we leverage K-Nearest Neighbors (KNN) to pinpoint the finger position relying on the fine-grained reflection features, and obtain a smoothed trace via Kalman filter. Additionally, we construct the reflection image of each multi-touch gesture from the reflection features by regarding the multiple fingers as a whole. Finally, we use a Convolutional Neural Network (CNN) to identify the multi-touch gestures based on the images. Extensive experiments validate that RF -finger can achieve as high as 88% and 92% accuracy for finger tracking and multi-touch gesture recognition, respectively.

Research on RFID Single Tag Contactless Gesture Recognition Based on Improved Convolutional Neural Network

WiFine: Real-Time Gesture Recognition Using Wi-Fi with Edge Intelligence

A RD-T Network for Hand Gesture Recognition Based on Millimeter-Wave Sensor

Hand gesture recognition scheme based on millimeter-wave radar with convolutional neural network

Gesture Recognition System Based on Neural Networks by Using COTS RFID Tag Array.

Multi - Touch in the Air: Device-Free Finger Tracking and Gesture Recognition Via COTS RFID.

Wear-free Gesture Recognition Based on Residual Features of RFID Signals

Research on Key Technologies of RFID Based Device Free Gesture Recognition

Gesture Recognition System Based on RFID.

Domain-Independent Gesture Recognition Using Single-Channel Time-Modulated Array

Towards Domain-independent Complex and Fine-grained Gesture Recognition with RFID.

RFnet: Automatic Gesture Recognition and Human Identification Using Time Series RFID Signals

GRfid: A Device-Free RFID-Based Gesture Recognition System

Gesture Recognition with RFID: an Experimental Study.

Doppler-Radar Based Hand Gesture Recognition System Using Convolutional Neural Networks

Dynamic Gesture Recognition Based on RF Sensor and AE-LSTM Neural Network

Intelligent Contactless Gesture Recognition Using WLAN Physical Layer Information

RF-Eletter: A Cross-Domain English Letter Recognition System Based on RFID

RF-CGR: Enable Chinese Character Gesture Recognition With RFID

Device-Free Gesture Recognition Using Time Series Rfid Signals

ReActor: Real-time and Accurate Contactless Gesture Recognition with RFID