Abstract:Background: Prompt engineering, focusing on crafting effective prompts to large language models (LLMs), has garnered attention for its capabilities at harnessing the potential of LLMs. This is even more crucial in the medical domain due to its specialized terminology and language technicity. Clinical natural language processing applications must navigate complex language and ensure privacy compliance. Prompt engineering offers a novel approach by designing tailored prompts to guide models in exploiting clinically relevant information from complex medical texts. Despite its promise, the efficacy of prompt engineering in the medical domain remains to be fully explored. Objective: The aim of the study is to review research efforts and technical approaches in prompt engineering for medical applications as well as provide an overview of opportunities and challenges for clinical practice. Methods: Databases indexing the fields of medicine, computer science, and medical informatics were queried in order to identify relevant published papers. Since prompt engineering is an emerging field, preprint databases were also considered. Multiple data were extracted, such as the prompt paradigm, the involved LLMs, the languages of the study, the domain of the topic, the baselines, and several learning, design, and architecture strategies specific to prompt engineering. We include studies that apply prompt engineering–based methods to the medical domain, published between 2022 and 2024, and covering multiple prompt paradigms such as prompt learning (PL), prompt tuning (PT), and prompt design (PD). Results: We included 114 recent prompt engineering studies. Among the 3 prompt paradigms, we have observed that PD is the most prevalent (78 papers). In 12 papers, PD, PL, and PT terms were used interchangeably. While ChatGPT is the most commonly used LLM, we have identified 7 studies using this LLM on a sensitive clinical data set. Chain-of-thought, present in 17 studies, emerges as the most frequent PD technique. While PL and PT papers typically provide a baseline for evaluating prompt-based approaches, 61% (48/78) of the PD studies do not report any nonprompt-related baseline. Finally, we individually examine each of the key prompt engineering–specific information reported across papers and find that many studies neglect to explicitly mention them, posing a challenge for advancing prompt engineering research. Conclusions: In addition to reporting on trends and the scientific landscape of prompt engineering, we provide reporting guidelines for future studies to help advance research in the medical field. We also disclose tables and figures summarizing medical prompt engineering papers available and hope that future contributions will leverage these existing works to better advance the field.

Promptwise: Prompt Engineering Paradigm for Enhanced Patient-Large Language Model Interactions Towards Medical Education

Prompt engineering on leveraging large language models in generating response to InBasket messages

Prompt engineering with a large language model to assist providers in responding to patient inquiries: a real-time implementation in the electronic health record

Prompt-Eng: Healthcare Prompt Engineering: Revolutionizing Healthcare Applications with Precision Prompts

Advanced Prompting As a Catalyst: Empowering Large Language Models in the Management of Gastrointestinal Cancers

Prompt Engineering Paradigms for Medical Applications: Scoping Review

Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances

An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study

Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs

Investigating the Impact of Prompt Engineering on the Performance of Large Language Models for Standardizing Obstetric Diagnosis Text: Comparative Study

Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy

Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial

Exploring Prompt Engineering Practices in the Enterprise

An Active Inference Strategy for Prompting Reliable Responses from Large Language Models in Medical Practice

PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models

Large Language Model Prompting Techniques for Advancement in Clinical Medicine

A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications

CoPrompter: User-Centric Evaluation of LLM Instruction Alignment for Improved Prompt Engineering

Do Physicians Know How to Prompt? The Need for Automatic Prompt Optimization Help in Clinical Note Generation