Abstract:Large Language Model (LLM) assistants, such as ChatGPT, have emerged as potential alternatives to search methods for helping users navigate complex, feature-rich software. LLMs use vast training data from domain-specific texts, software manuals, and code repositories to mimic human-like interactions, offering tailored assistance, including step-by-step instructions. In this work, we investigated LLM-generated software guidance through a within-subject experiment with 16 participants and follow-up interviews. We compared a baseline LLM assistant with an LLM optimized for particular software contexts, SoftAIBot, which also offered guidelines for constructing appropriate prompts. We assessed task completion, perceived accuracy, relevance, and trust. Surprisingly, although SoftAIBot outperformed the baseline LLM, our results revealed no significant difference in LLM usage and user perceptions with or without prompt guidelines and the integration of domain context. Most users struggled to understand how the prompt's text related to the LLM's responses and often followed the LLM's suggestions verbatim, even if they were incorrect. This resulted in difficulties when using the LLM's advice for software tasks, leading to low task completion rates. Our detailed analysis also revealed that users remained unaware of inaccuracies in the LLM's responses, indicating a gap between their lack of software expertise and their ability to evaluate the LLM's assistance. With the growing push for designing domain-specific LLM assistants, we emphasize the importance of incorporating explainable, context-aware cues into LLMs to help users understand prompt-based interactions, identify biases, and maximize the utility of LLM assistants.

Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs through Conversations, Toolbars, and Prompts

Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances

Rethinking ChatGPT's Success: Usability and Cognitive Behaviors Enabled by Auto-regressive LLMs' Prompting

Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting

Large Language Models Meet User Interfaces: The Case of Provisioning Feedback

Task Supportive and Personalized Human-Large Language Model Interaction: A User Study

Negotiating with LLMS: Prompt Hacks, Skill Gaps, and Reasoning Deficits

Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation

Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking

The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development

User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants

Human-Centered LLM-Agent User Interface: A Position Paper

Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers

Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Using an LLM to Help With Code Understanding

Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration

On the Robustness of Editing Large Language Models

Towards Full Authorship with AI: Supporting Revision with AI-Generated Views

Deceptive Patterns of Intelligent and Interactive Writing Assistants