Abstract:Trust in automation is often strongly tied to an agent’s performance. However, our understanding of imperfect agents’ behaviours and its impact on trust is limited. In this paper, we study the relationship between performance, reliance and trust in a set of human-agent collaborative tasks. Participants collaborated with different automated agents that performed similarly but made errors in different ways; namely mistakes (error of prioritization), lapses (error of omission) and slips (lowered accuracy). We conducted a 4x2 within-subjects experiment (n=24) varying the agent behaviours (no error, slips, mistakes and lapses) and task difficulty (easy/hard) during a real-time collaborative game. Our results show that, at the same level of agent performance, agents’ errors are perceived differently and change the way participants interact with agents. For instance, slips and mistakes are more harmful to performance than lapses while slips are more harmful to reliance than mistakes.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to understand the impact of different types of automated agent errors on trust, dependence, and performance in human - agent collaboration. Specifically, the research aims to explore the following issues:
1. **The impact of different types of agent errors on human participants**: The research focuses on three main types of agent errors - mistakes, lapses, and slips, and examines how these errors affect participants' task performance, their degree of dependence on the agent, and their trust in the agent.
2. **The relationship between trust and performance**: The research attempts to verify whether different types of errors will be perceived differently by participants at the same level of agent performance, and how these perceptual differences change the way participants interact with the agent.
3. **Designing more effective collaborative systems**: By understanding the unique impacts of different error types, the research hopes to provide guidance for designing safer and more reliable automated systems to avoid human behavior deviations caused by agent errors, such as misusing or abandoning agent functions.
### Research Background
Trust is crucial in human - machine collaboration, especially in cases where automated agents have imperfect behaviors. Previous research has shown that an agent's performance is one of the key factors affecting human trust, but there is still a lack of in - depth understanding of the specific impacts of different types of agent errors. Therefore, through experimental design, this research systematically manipulates the behavior of agents to explore the specific impacts of different error types on trust, dependence, and performance.
### Research Methods
The research adopts a 4×2 within - subjects design, where the four independent variables are different behaviors of the agent (no error, slip, mistake, lapse), and two difficulty levels (easy and difficult). A total of 24 participants completed the experiment. They cooperated with different automated agents in a real - time collaborative game to complete the target - destruction task. During the experiment, data such as participants' task performance, control time of the agent, and number of corrections were recorded, and participants' cognitive load and trust in the agent were evaluated through questionnaires.
### Main Findings
- **Performance differences**: Different types of agent errors have a significant impact on task performance. For example, slips and mistakes have a more negative impact on performance than lapses.
- **Dependence changes**: The degree of participants' dependence on the agent varies depending on the error type. Slips affect participants' dependence more seriously than mistakes.
- **Trust perception**: Participants have the highest trust in error - free agents, and their trust in error - making agents is significantly reduced, especially in agents with mistakes and slips.
### Conclusion
The research results show that when designing human - machine collaborative systems, agents should be made to avoid showing hesitant or inaccurate information as much as possible, because such errors will significantly reduce user trust and dependence. In contrast, a complete lack of input (such as a lapse) may be more acceptable and easier for users to correct than inaccurate input. This finding provides an important reference for designing more reliable and user - friendly automated systems in the future.