Abstract:To prevent users’ privacy from leakage, more and more mobile devices employ biometric-based authentication approaches, such as fingerprint, face recognition, voiceprint authentications, and so on, to enhance the privacy protection. However, these approaches are vulnerable to replay attacks. Although the state-of-art solutions utilize liveness verification to combat the attacks, existing approaches are sensitive to ambient environments, such as ambient lights and surrounding audible noises. Toward this end, we explore liveness verification of user authentication leveraging users’ mouth movements, which are robust to noisy environments. In this paper, we propose a lip reading-based user authentication system, LipPass, which extracts unique behavioral characteristics of users’ speaking mouths through acoustic sensing on smartphones for user authentication. We first investigate Doppler profiles of acoustic signals caused by users’ speaking mouths and find that there are unique mouth movement patterns for different individuals. To characterize the mouth movements, we propose a deep learning-based method to extract efficient features from Doppler profiles and employ softmax function, support vector machine, and support vector domain description to construct multi-class identifier, binary classifiers, and spoofer detectors for mouth state identification, user identification, and spoofer detection, respectively. Afterward, we develop a balanced binary tree-based authentication approach to accurately identify each individual leveraging these binary classifiers and spoofer detectors with respect to registered users. Through extensive experiments involving 48 volunteers in four real environments, LipPass can achieve 90.2% accuracy in user identification and 93.1% accuracy in spoofer detection.

ProxiTalk

<i>ChestLive</i>: Fortifying Voice-based Authentication with Chest Motion Biometric on Smart Devices

LVID: A Multimodal Biometrics Authentication System on Smartphones.

VoicePop: A Pop Noise based Anti-spoofing System for Voice Authentication on Smartphones

ProxiMic: Convenient Voice Activation via Close-to-Mic Speech Detected by a Single Microphone

PrivateTalk: Activating Voice Input with Hand-On-Mouth Gesture Detected by Bluetooth Earphones

Lip-Interact: Improving Mobile Device Interaction with Silent Speech Commands.

Lip Reading-Based User Authentication Through Acoustic Sensing on Smartphones.

LipPass: Lip Reading-based User Authentication on Smartphones Leveraging Acoustic Signals.

Enabling Private and Non-Intrusive Smartphone Calls with LipTalk

Exploring Interactive Gestures with Voice Assistant on HMDs in Social Situations

EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users.

EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios

Combating Voice Spoofing Attacks on Wearables Via Speech Movement Sequences

Enabling Voice-Accompanying Hand-to-Face Gesture Recognition with Cross-Device Sensing

Poster - Sonicnect: Accurate Hands-Free Gesture Input System With Smart Acoustic Sensing

A New Mmwave-Speech Multimodal Speech System for Voice User Interface

Building Proactive Voice Assistants: When and How (not) to Interact

A Continuous Liveness Detection for Voice Authentication on Smart Devices

Robust Dual-Modal Speech Keyword Spotting for XR Headsets

Hidebehind: Enjoy Voice Input with Voiceprint Unclonability and Anonymity.