I Know What You Enter on Gear VR

Zhen Ling,Zupei Li,Chen Chen,Junzhou Luo,Wei Yu,Xinwen Fu
DOI: https://doi.org/10.1109/CNS.2019.8802674
2019-01-01
Abstract:Virtual reality (VR) techniques offer users immersive experiences in a virtual environment (VE) where they can enjoy gaming, training, shopping and social activities. However, security and privacy issues of VR are rarely investigated. In this paper, we introduce novel computer vision-based and motion sensor-based side channel attacks to infer keystrokes in a virtual environment. In the computer vision-based attack, a stereo camera records a user maneuvering a pointing device and inputting passwords. Various computer vision techniques are used to detect the touching frames in which the user taps a touchpad to enter keys on the virtual keyboard within the VE. We then estimate the pose of the headset and derive the orientation of the virtual pointing ray in each touching frame. In the motion sensor-based attack, we exploit sensors including accelerometer, gyroscope and magnetometer built inside the pointing device to retrieve orientation angles of the pointing device. With orientation angles of the pointing device, we can derive the empirical rotation angles and infer the clicked keys by assessing the similarity between the empirical rotation angles and the reference ones between keys. Extensive real-world experiments are performed to demonstrate the feasibility and effectiveness of these side channel attacks.
What problem does this paper attempt to address?