Investigating Interaction Modes and User Agency in Human-LLM Collaboration for Domain-Specific Data Analysis

Jiajing Guo,Vikram Mohanty,Jorge Piazentin Ono,Hongtao Hao,Liang Gou,Liu Ren

DOI: https://doi.org/10.1145/3613905.3651042

2024-05-09

Abstract:Despite demonstrating robust capabilities in performing tasks related to general-domain data-operation tasks, Large Language Models (LLMs) may exhibit shortcomings when applied to domain-specific tasks. We consider the design of domain-specific AI-powered data analysis tools from two dimensions: interaction and user agency. We implemented two design probes that fall on the two ends of the two dimensions: an open-ended high agency (OHA) prototype and a structured low agency (SLA) prototype. We conducted an interview study with nine data scientists to investigate (1) how users perceived the LLM outputs for data analysis assistance, and (2) how the two test design probes, OHA and SLA, affected user behavior, performance, and perceptions. Our study revealed insights regarding participants' interactions with LLMs, how they perceived the results, and their desire for explainability concerning LLM outputs, along with a noted need for collaboration with other users, and how they envisioned the utility of LLMs in their workflow.

Human-Computer Interaction

What problem does this paper attempt to address?

### Problems the paper attempts to solve

This paper aims to explore the application of large - language models (LLM) in domain - specific data analysis tasks, especially to design AI - assisted data analysis tools from two dimensions - interaction mode and user agency. By implementing two design probes, one with an open - ended high - agency (OHA) workflow and the other with a structured low - agency (SLA) workflow, the paper studies users' perception of LLM outputs and the impact of different design patterns on user behavior, performance, and perception.

Specifically, the paper attempts to answer the following research questions:
1. **RQ1**: How do users perceive the outputs of LLM in domain - specific data analysis tasks?
2. **RQ2**: What kind of design is most suitable for LLM - assisted data analysis tools?

### Research background and motivation

Although large - language models (LLM) show strong capabilities in general - domain data manipulation tasks, they may have deficiencies in domain - specific tasks. This is because although the extensive training data enables LLM to understand a wide range of concepts or terms, they may not be able to understand certain organization - specific meanings or subtle contexts.

To explore this issue, the author designed two different prototype tools from two dimensions - interaction mode and user agency:
- **OHA (Open - ended High Agency)**: It has an open - ended workflow and gives users high agency at the execution and planning levels.
- **SLA (Structured Low Agency)**: It has a structured workflow, guides users to complete tasks step by step, and gives users lower execution and planning agency.

### Research methods

The author conducted an interview study and invited nine data scientists to use these two prototype tools to complete two domain - specific data analysis tasks. The study was divided into three parts:
1. **Pre - study interview**: To understand the participants' backgrounds and relevant work experiences.
2. **Data analysis tasks**: Participants used OHA and SLA to complete two data analysis tasks and were encouraged to "think aloud".
3. **Criticism and reflection**: Participants compared the two tools and discussed their advantages, disadvantages, and potential uses.

### Main findings

1. **Users' perception of LLM outputs**:
- **Output quality**: Participants were satisfied with OHA's ability to generate initial code, although it sometimes made mistakes. The charts and explanations generated by SLA were also considered helpful for communication with stakeholders.
- **Trust and verifiability**: Users were cautious about the results of SLA, mainly because of the lack of verification methods and the consistency of results. OHA users also expressed similar trust issues.
- **Need for explanation**: Users emphasized the importance of transparency, especially when the reasoning process of LLM was not clear.

2. **The impact of design patterns on user behavior**:
- **Interaction mode**: In the OHA mode, experienced users tended to provide sample data or key columns to provide context. In the SLA mode, users relied on the system's structure to guide the data analysis process.
- **User agency**: In the OHA setting, users showed high agency and independently decided the progress of the analysis. SLA reduced user agency by providing detailed domain knowledge and analysis steps.

### Discussion

1. **Design considerations for AI - assisted domain - specific data analysis**:
- **Structured interaction**: A structured workflow can help users understand the functions of LLM and improve transparency. A design that combines open - ended and structured interactions may be beneficial.
- **User agency at the execution level**: Automated code generation, debugging, and result explanation are welcomed, but users hope to be able to modify the code before and after execution.
- **User agency at the planning level**: Users showed interest and caution in assistance at the planning level, emphasizing that the final decision should be made by the users themselves.

2. **Adjusting user trust and expectations**:
- **AI explainability**: Providing a verification sandbox and step - by - step explanation of the analysis process can enhance users' trust and understanding.

### Conclusion

This study explored the application of LLM in domain - specific data analysis through two different design probes. The study found that although users...