Exploring Smart Agents for the Interaction with Multimodal Mediated Environments

Robert Richer,Nan Zhao,Bjoern M. Eskofier,Joseph A. Paradiso

DOI: https://doi.org/10.3390/mti4020027

2020-06-06

Multimodal Technologies and Interaction

Abstract:After conversational agents have been made available to the broader public, we speculate that applying them as a mediator for adaptive environments reduces control complexity and increases user experience by providing a more natural interaction. We implemented and tested four agents, each of them differing in their system intelligence and input modality, as personal assistants for Mediated Atmospheres, an adaptive smart office prototype. They were evaluated in a user study ( N = 33 ) to collect subjective and objective measures. Results showed that a smartphone application was the most favorable system, followed by conversational text and voice agents that were perceived as being more engaging and intelligent than a non-conversational voice agent. Significant differences were observed between native and non-native speakers in both subjective and objective measures. Our findings reveal the potential of conversational agents for the interaction with adaptive environments to reduce work and information overload.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to simplify the interaction between users and multimodal mediated environments by applying smart agents, thereby reducing control complexity and enhancing the user experience. Specifically, the authors hope to enable users to interact more naturally with the adaptive environment by introducing smart agents as intermediaries, thus alleviating work and information overload.

### Key Problems and Background

1. **Existing Problems**:
- Current adaptive environments (such as smart offices) usually require users to manually adjust environmental parameters (such as lighting, sound, etc.), which is both cumbersome and easily distracts the user's attention.
- Although traditional graphical user interfaces (GUIs) are powerful, they are not intuitive or natural enough in some cases.

2. **Solution Concepts**:
- Introduce smart agents, especially conversational agents, to provide a more natural interaction method.
- These agents can communicate with users through multiple input modalities such as voice and text, and automatically adjust environmental settings according to the user's preferences and the current situation.

### Specific Research Objectives

- **Evaluate Different Types of Smart Agents**: The paper designed and tested four different smart agents, which differ in system intelligence level and input modality (such as basic voice agents, advanced voice agents, advanced text agents, and smartphone applications).
- **Compare User Experiences**: Through user experiments, collect subjective and objective data to compare the differences among these agents in terms of user experience, intelligence perception, and engagement.
- **The Influence of Language Background**: The study found out whether there are significant differences between native and non - native speakers when using these agents, especially in voice interaction.

### Research Significance

Through this research, the authors hope to reveal the potential of conversational smart agents in adaptive environments, especially in the smart office scenario, to help users manage their workspaces more efficiently, and improve productivity and satisfaction. At the same time, the research results also provide valuable references for the future design of smart agents, especially in terms of multimodal interaction and user - friendliness.

### Conclusions

The main conclusion of the paper is that although smartphone applications are still the most popular system, conversational agents (especially advanced voice and text agents) are considered more attractive and intelligent. In addition, the study also found significant differences between native and non - native speakers when using these agents, which provides an important basis for future multilingual support design.