Abstract:Motion sensors on current smartphones have been exploited for audio eavesdropping due to their sensitivity to vibrations. However, this threat is considered low-risk because of two widely acknowledged limitations: First, unlike microphones, motion sensors can only pick up speech signals traveling through a solid medium. Thus, the only feasible setup reported previously is to use a smartphone gyroscope to eavesdrop on a loudspeaker placed on the same table. The second limitation comes from a common sense that these sensors can only pick up a narrow band (85-100Hz) of speech signals due to a sampling ceiling of 200Hz. In this paper, we revisit the threat of motion sensors to speech privacy and propose AccelEve, a new side-channel attack that employs a smartphone's accelerometer to eavesdrop on the speaker in the same smartphone. Specifically, it utilizes the accelerometer measurements to recognize the speech emitted by the speaker and to reconstruct the corresponding audio signals. In contrast to previous works, our setup allows the speech signals to always produce strong responses in accelerometer measurements through the shared motherboard, which successfully addresses the first limitation and allows this kind of attacks to penetrate into real-life scenarios. Regarding the sampling rate limitation, contrary to the widely-held belief, we observe up to 500Hz sampling rates in recent smartphones, which almost covers the entire fundamental frequency band (85-255Hz) of adult speech. On top of these pivotal observations, we propose a novel deep learning based system that learns to recognize and reconstruct speech information from the spectrogram representation of acceleration signals. This system employs adaptive optimization on deep neural networks with skip connections using robust and generalizable losses to achieve robust recognition and reconstruction performance. Extensive evaluations demonstrate the effectiveness and high accuracy of our attack under various settings.

DSP.Ear: Leveraging Co-Processor Support for Continuous Audio Sensing on Smartphones

Speakersense: Energy Efficient Unobtrusive Speaker Identification On Mobile Phones

In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms

A High-Performance, Low Power Research Hearing Aid featuring a High-Level Programmable Custom 22nm FDSOI SoC

System Architecture of a Smart Binaural Hearing Aid Using a Mobile Computing Platform

A Smart Binaural Hearing Aid Architecture Leveraging a Smartphone APP with Deep-Learning Speech Enhancement.

A Smart Binaural Hearing Aid Architecture Based on a Mobile Computing Platform

Improving Energy Efficiency Of Personal Sensing Applications With Heterogeneous Multi-Processors

EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers

Enabling Energy Efficient Continuous Sensing on Mobile Phones with LittleRock

EERS : Energy Efficient Responsive Sleeping on Mobile Phones

EarIO: A Low-power Acoustic Sensing Earable for Continuously Tracking Detailed Facial Movements.

A 79-Db SNR 1.1-Mw Fully Integrated Hearing Aid SoC

AdaOper: Energy-efficient and Responsive Concurrent DNN Inference on Mobile Devices

InertiEAR: Automatic and Device-independent IMU-based Eavesdropping on Smartphones

Device-Independent Smartphone Eavesdropping Jointly Using Accelerometer and Gyroscope

Optimization of ETSI DSR frontend software on a high-efficient audio DSP

DyPS: Dynamic Processor Switching for Energy-Aware Video Decoding on Multi-core SoCs

Messages Behind the Sound: Real-Time Hidden Acoustic Signal Capture with Smartphones

A Wearable ECG Monitoring Device with Flexible Embedded Denoising and Compression

Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer