Gagan Bansal,Jennifer Wortman Vaughan,Saleema Amershi,Eric Horvitz,Adam Fourney,Hussein Mozannar,Victor Dibia,Daniel S. Weld
Abstract:Remarkable advancements in modern generative foundation models have enabled the development of sophisticated and highly capable autonomous agents that can observe their environment, invoke tools, and communicate with other agents to solve problems. Although such agents can communicate with users through natural language, their complexity and wide-ranging failure modes present novel challenges for human-AI interaction. Building on prior research and informed by a communication grounding perspective, we contribute to the study of \emph{human-agent communication} by identifying and analyzing twelve key communication challenges that these systems pose. These include challenges in conveying information from the agent to the user, challenges in enabling the user to convey information to the agent, and overarching challenges that need to be considered across all human-agent communication. We illustrate each challenge through concrete examples and identify open directions of research. Our findings provide insights into critical gaps in human-agent communication research and serve as an urgent call for new design patterns, principles, and guidelines to support transparency and control in these systems.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the challenges faced by modern intelligent agents (AI agents) when communicating with human users. With the development of generative foundation models, these intelligent agents have become more complex and powerful, capable of performing a wide range of tasks, but at the same time, they have also brought new and more complex communication problems. Specifically, the paper focuses on the following aspects:
1. **Challenges in Information Conveyance**:
- **Challenges from Agent to User**: How can the agent effectively convey its actions, intentions, and results to the user? For example, how should the agent help the user verify whether its actions meet expectations (X1), how to ensure the consistency of actions (X2), and how to select an appropriate level of information detail (X3).
- **Challenges from User to Agent**: How can the user effectively convey their goals, preferences, and constraints to the agent? For example, how should the user express which preferences need to be respected (U1), and how should the agent adjust its behavior according to user feedback (U3).
2. **General Communication Difficulties**:
- There are also some general challenges in the communication between the agent and the user, such as how to consider the past interaction history to optimize the current communication (X4). In addition, problems such as how to avoid inconsistency and reduce the user's cognitive burden also need to be solved.
### Specific Problem Summary
- **X1: How can the agent help the user verify its actions?**
Although modern agents are powerful, they are not perfect. When the agent performs a task, errors may occur, especially for complex or multi - step tasks. Therefore, a mechanism needs to be designed to enable the user to easily verify the agent's actions to ensure that both parties have a common understanding of the task.
- **X2: How can the agent convey consistent behavior?**
The agent's behavior may appear inconsistent due to the randomness of the underlying model and environmental changes. This inconsistency will prevent the user from building an accurate mental model, which in turn will affect trust. Therefore, it is necessary to ensure that the agent's behavior is as consistent as possible or at least appears consistent to the user.
- **X3: How should the agent select an appropriate level of information detail?**
Providing too much information may overburden the user, while providing too little information may lead to misunderstandings. Therefore, the agent needs to find a balance between the two, so that the user can understand its actions without causing information overload.
- **X4: What past interactions should the agent consider during communication?**
Context is crucial for communication. The agent should consider which past interactions are helpful for the current communication in order to better understand and meet the user's needs.
By solving these problems, the paper aims to provide guidance for the future design of intelligent agents to support transparency and controllability, thereby achieving more effective collaboration.