Abstract:Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the general public, including individuals with no prior technical experience in NLP techniques. However, natural language prompts can vary significantly in terms of their linguistic structure, context, and other semantics. Modifying one or more of these aspects can result in significant differences in task performance. Non-expert users may find it challenging to identify the changes needed to improve a prompt, especially when they lack domain-specific knowledge and lack appropriate feedback. To address this challenge, we present PromptAid, a visual analytics system designed to interactively create, refine, and test prompts through exploration, perturbation, testing, and iteration. PromptAid uses multiple, coordinated visualizations which allow users to improve prompts by using the three strategies: keyword perturbations, paraphrasing perturbations, and obtaining the best set of in-context few-shot examples. PromptAid was designed through an iterative prototyping process involving NLP experts and was evaluated through quantitative and qualitative assessments for LLMs. Our findings indicate that PromptAid helps users to iterate over prompt template alterations with less cognitive overhead, generate diverse prompts with help of recommendations, and analyze the performance of the generated prompts while surpassing existing state-of-the-art prompting interfaces in performance.

Large Language Models and Applications: The Rebirth of Enterprise Knowledge Management and the Rise of Prompt Libraries

Large Language Models and Knowledge Graphs: Opportunities and Challenges

The Rise and Design of Enterprise Large Language Models

Prompt-Time Ontology-Driven Symbolic Knowledge Capture with Large Language Models

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

Exploring Prompt Engineering Practices in the Enterprise

Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts

A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications

Application of large language models in professional fields

Prompting Frameworks for Large Language Models: A Survey

PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models

The Prompt Canvas: A Literature-Based Practitioner Guide for Creating Effective Prompts in Large Language Models

Ask more, know better: Reinforce-Learned Prompt Questions for Decision Making with Large Language Models

Prompt Engineering: Guiding the Way to Effective Large Language Models

Prompt Design and Engineering: Introduction and Advanced Methods

Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review

An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide

Exploring the Capabilities of Prompted Large Language Models in Educational and Assessment Applications

Prompt-Time Symbolic Knowledge Capture with Large Language Models

KnowGPT: Knowledge Graph based Prompting for Large Language Models

Large Language Models Are Also Good Prototypical Commonsense Reasoners