ViViSnoop: Someone is Snooping Your Typing Without Seeing It!

Kun Jin,Si Fang,Chunyi Peng,Zhiyang Teng,Xufei Mao,Lan Zhang,Xiangyang Li

DOI: https://doi.org/10.1109/cns.2017.8228624

2017-01-01

Abstract:In the paper, we present ViViSnoop, a novel video-assisted keystroke inference attack which snoops the victim user's typed input without visually seeing it. Instead, it infers the typed input from the vibration extracted from the video capturing the desk where the physical or virtual keyboard is placed. ViViSnoop is built on the fact that keystrokes on the desk incur mechanical vibrations which are subtle but still extractable through computer vision processing. To further exploit subtle vibration patterns for accurate and reliable keystroke inference, ViViSnoop incorporates a suite of techniques such as vibration-specific video processing to enhance raw vibration quality, a novel virtual sensing array technique to develop fine-grained location signatures, and a two-phase classifier to achieve high accuracy and efficiency. Our extensive evaluation shows that ViViSnoop realizes high-accuracy keystroke inference. ViViSnoop achieves around 55% single-character accuracy in inferring passwords (random inputs). In word and sentence inference, it achieves 71.4% and 59.4% accuracy using top-1 choices and even almost 100% and 75% using top-10 choices. This means there is almost no difficulty in understanding user inputs. It imposes a covert and serious threat to leak user typing in public/office spaces.