Abstract:SpeeT is an interactive system implementing an approach for combining touch gestures with speech in automotive environments, exploiting the specific advantages of each modality. The main component of the implemented prototype is a speech-enabled, multi-touch steering wheel. A microphone recognizes speech commands while a wheel-integrated tablet allows touch gestures to be recognized. Using this steering wheel, the driver can control objects of the simulated car environment (e.g., windows, cruise control). The idea is to use the benefits of both interaction styles and to overcome the problems of each single interaction style. While touch input is well suited for controlling functions, speech is powerful to select specific objects from a large pool of items. The combination simplifies the problem of remembering possible speech commands by two means: (1) speech is used to specify objects or functionalities and (2) in smart environments - particularly in cars - interaction objects are visible to the user and do not need to be remembered. Our approach is specifically designed to support important rules in UI design, namely: provide feedback, support easy reversal of action, reduce memory load, and make opportunities for action visible.

SpeeT: A Multimodal Interaction Style Combining Speech and Touch Interaction in Automotive Environments