When camera meets accelerometer: a novel way for 3d interaction of mobile phone

Mingming Fan,Donald Patterson,Yuanchun Shi

DOI: https://doi.org/10.1145/2371664.2371689

2012-01-01

Abstract:Performing real-time 3D motion interaction with a mobile phone is useful for extending interaction out of a limited screen. However, current phone motion detection approaches, which use only camera or accelerometer, have limitations in recognizing versatile 3D motions simultaneously. In this paper we present a real-time approach designed for 3D tasks by holding and moving a phone equipped with both a camera and an accelerometer. By analyzing both motion features from image frames and changes of accelerometer data simultaneously, our approach can naturally distinguish translation and rotation of a mobile phone. In a pilot user study, we demonstrated our design requires low learning effort and has high accuracy. Since it does not require any outside infrastructure, our approach can be applied to any phone with a camera and an accelerometer. Our main contribution is giving mobile phone users 3D interaction ability by exploiting hand translation and rotation.