VisaudiBlow: Fine-grained Blowing Interaction Based on Visual and Auditory Detection.

Yuxuan Chen,Yi Jin,Zhaoning Zeng,Shiyu Chen,Yang Zhou,Xinchi Xu,Jie Liu,Guihuan Feng
DOI: https://doi.org/10.1145/3544549.3585829
2023-01-01
Abstract:More and more studies regard blowing as a means of reflecting the user’s physiological conditions. Its potential as an independent input channel has not been fully explored. In this paper, we analyze the design space of blowing as an independent interactive channel and propose a novel blowing recognition approach, VisaudiBlow, based on visual and audio detection. VisaudiBlow uses Facial Tracker to capture the mouth expression when doing specific blowing actions, and audio information generated by blowing is obtained by a microphone beside the mouth, enabling identification and detection of the input parameters such as blowing type, direction, and intensity. In order to verify the validity of blowing as an independent input channel, a prototype system BlowUI is implemented. The experimental results demonstrate that blow interaction can effectively complete the menu control tasks and achieve the same interaction efficiency as controllers in specific scenarios.
What problem does this paper attempt to address?