Abstract:This paper aims to develop a semi-formal design space for Human-AI interactions, by building a set of interaction primitives which specify the communication between users and AI systems during their interaction. We show how these primitives can be combined into a set of interaction patterns which can provide an abstract specification for exchanging messages between humans and AI/ML models to carry out purposeful interactions. The motivation behind this is twofold: firstly, to provide a compact generalisation of existing practices, that highlights the similarities and differences between systems in terms of their interaction behaviours; and secondly, to support the creation of new systems, in particular by opening the space of possibilities for interactions with models. We present a short literature review on frameworks, guidelines and taxonomies related to the design and implementation of HAI interactions, including human-in-the-loop, explainable AI, as well as hybrid intelligence and collaborative learning approaches. From the literature review, we define a vocabulary for describing information exchanges in terms of providing and requesting particular model-specific data types. Based on this vocabulary, a message passing model for interactions between humans and models is presented, which we demonstrate can account for existing systems and approaches. Finally, we build this into design patterns as mid-level constructs that capture common interactional structures. We discuss how this approach can be used towards a design space for Human-AI interactions that creates new possibilities for designs as well as keeping track of implementation issues and concerns.

What problem does this paper attempt to address?

The paper aims to address the following issues: 1. **Defining Interaction Primitives**: The goal of the paper is to develop a semi-formal design space for Human-AI Interaction (HAI) by constructing a set of interaction primitives to specify how users communicate with AI systems during the interaction process. These primitives can be combined into a series of interaction patterns, providing an abstract specification to exchange information between humans and AI/ML models for meaningful interaction. 2. **Generalization of Existing Practices**: The paper attempts to provide a concise generalization of existing practices, highlighting the similarities and differences in interaction behaviors across different systems, thereby supporting the creation of new systems, particularly by expanding the space of possibilities to enhance interaction with models. 3. **Literature Review**: The paper provides a literature review of frameworks, guidelines, and taxonomies related to the design and implementation of human-computer interaction, including Human-in-the-Loop (HITL), Explainable AI (XAI), as well as Hybrid Intelligence (HI) and collaborative learning approaches. 4. **Defining a Vocabulary**: Based on the literature review, the authors define a vocabulary to describe information exchange, involving the provision and request of specific model data types. 5. **Messaging Model**: Based on the aforementioned vocabulary, a messaging model for interaction between humans and models is proposed, demonstrating how it can interpret existing systems and methods. 6. **Design Pattern Construction**: Finally, these primitives are constructed into design patterns as mid-level constructs, capturing common interaction structures and discussing how this approach can create a design space for human-AI interaction, thereby creating new design possibilities while also tracking implementation issues and concerns. The core of the paper lies in proposing a design space derived from specific user-model interaction actions, enabling designers and developers to more directly engage with the possibilities of model interaction by providing a series of possibilities to map human concepts to specific communication information. This approach attempts to bridge the gap between high-level frameworks and practical design practices, allowing more people to participate in the design process of HAI interactions.

Unpacking Human-AI interactions: From interaction primitives to a design space

Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation

Deconstructing Human‐AI Collaboration: Agency, Interaction, and Adaptation

The Interaction Design of 3D Virtual Humans: A Survey

Grasping AI: experiential exercises for designers

A Framework to Study Human-AI Collaborative Design Space Exploration

Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review

Human-AI Teamwork Interface Design Using Patterns of Interactions

Re-examining Whether, Why, and How Human-AI Interaction is Uniquely Difficult to Design

Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks

A Mental-Model Centric Landscape of Human-AI Symbiosis

Guidelines for Human-AI Interaction

Mapping the Design Space of Human-AI Interaction in Text Summarization

Characterizing and modeling harms from interactions with design patterns in AI interfaces

An HCAI Methodological Framework: Putting It Into Action to Enable Human-Centered AI

Enabling Human-Centered AI: A Methodological Perspective

eXtended Artificial Intelligence: New Prospects of Human-AI Interaction Research

Intelligent interaction design: the role of human‐computer interaction research in the design of intelligent systems

Human-AI interaction research agenda: A user-centered perspective

Human-AI Interaction Design in Machine Teaching

Designing AI to Work WITH or FOR People?