Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summaries

Peter Du,Surya Murthy,Katherine Driggs-Campbell

2023-04-02

Abstract:As advances in artificial intelligence enable increasingly capable learning-based autonomous agents, it becomes more challenging for human observers to efficiently construct a mental model of the agent's behaviour. In order to successfully deploy autonomous agents, humans should not only be able to understand the individual limitations of the agents but also have insight on how they compare against one another. To do so, we need effective methods for generating human interpretable agent behaviour summaries. Single agent behaviour summarization has been tackled in the past through methods that generate explanations for why an agent chose to pick a particular action at a single timestep. However, for complex tasks, a per-action explanation may not be able to convey an agents global strategy. As a result, researchers have looked towards multi-timestep summaries which can better help humans assess an agents overall capability. More recently, multi-step summaries have also been used for generating contrasting examples to evaluate multiple agents. However, past approaches have largely relied on unstructured search methods to generate summaries and require agents to have a discrete action space. In this paper we present an adaptive search method for efficiently generating contrasting behaviour summaries with support for continuous state and action spaces. We perform a user study to evaluate the effectiveness of the summaries for helping humans discern the superior autonomous agent for a given task. Our results indicate that adaptive search can efficiently identify informative contrasting scenarios that enable humans to accurately select the better performing agent with a limited observation time budget.

Robotics,Artificial Intelligence,Multiagent Systems

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address how humans can efficiently understand the behavior capabilities of multiple autonomous agents and accurately determine which agent performs better in a specific task within a limited time. With the advancement of artificial intelligence, the capabilities of autonomous agents are becoming increasingly powerful, making it difficult for human observers to quickly construct mental models of these agents' behaviors. For the successful deployment of autonomous agents, humans need to understand not only the limitations of a single agent but also be able to compare the strengths and weaknesses of different agents.

Specifically, the paper focuses on the following points:

1. **Limitations of Single-Agent Behavior Summarization**: Existing methods typically generate behavior summaries by explaining why an agent chose a specific action at a particular time step. However, for complex tasks, the explanation of a single action may not convey the agent's overall strategy.

2. **Need for Multi-Step Summarization**: Researchers have turned to multi-step summarization to better help humans assess the overall capabilities of an agent. However, past methods mostly rely on unstructured search methods and require agents to have discrete action spaces.

3. **Challenges of Comparative Behavior Summarization**: When comparing multiple agents, generating meaningful comparative summaries is particularly challenging. Existing methods lack structured approaches when exploring the scenario space and are not suitable for agents with continuous action spaces.

To address these issues, the paper proposes an adaptive search method that can efficiently generate comparative behavior summaries, supporting agents with continuous state and action spaces. Through user studies, the effectiveness of this method is validated, demonstrating its ability to help humans accurately choose the better-performing agent within a limited observation time.