Inhibitors of prostaglandin, prostacyclin, and thromboxane synthesis.

J. Vane

Abstract:

What problem does this paper attempt to address?

An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study

Sonish Sivarajkumar,Mark Kelley,Alyssa Samolyk-Mazzanti,Shyam Visweswaran,Yanshan Wang

DOI: https://doi.org/10.2196/55318

IF: 3.2

2024-04-09

JMIR Medical Informatics

Abstract:Background: Large language models (LLMs) have shown remarkable capabilities in natural language processing (NLP), especially in domains where labeled data are scarce or expensive, such as the clinical domain. However, to unlock the clinical knowledge hidden in these LLMs, we need to design effective prompts that can guide them to perform specific clinical NLP tasks without any task-specific training data. This is known as in-context learning, which is an art and science that requires understanding the strengths and weaknesses of different LLMs and prompt engineering approaches. Objective: The objective of this study is to assess the effectiveness of various prompt engineering techniques, including 2 newly introduced types—heuristic and ensemble prompts, for zero-shot and few-shot clinical information extraction using pretrained language models. Methods: This comprehensive experimental study evaluated different prompt types (simple prefix, simple cloze, chain of thought, anticipatory, heuristic, and ensemble) across 5 clinical NLP tasks: clinical sense disambiguation, biomedical evidence extraction, coreference resolution, medication status extraction, and medication attribute extraction. The performance of these prompts was assessed using 3 state-of-the-art language models: GPT-3.5 (OpenAI), Gemini (Google), and LLaMA-2 (Meta). The study contrasted zero-shot with few-shot prompting and explored the effectiveness of ensemble approaches. Results: The study revealed that task-specific prompt tailoring is vital for the high performance of LLMs for zero-shot clinical NLP. In clinical sense disambiguation, GPT-3.5 achieved an accuracy of 0.96 with heuristic prompts and 0.94 in biomedical evidence extraction. Heuristic prompts, alongside chain of thought prompts, were highly effective across tasks. Few-shot prompting improved performance in complex scenarios, and ensemble approaches capitalized on multiple prompt strengths. GPT-3.5 consistently outperformed Gemini and LLaMA-2 across tasks and prompt types. Conclusions: This study provides a rigorous evaluation of prompt engineering methodologies and introduces innovative techniques for clinical information extraction, demonstrating the potential of in-context learning in the clinical domain. These findings offer clear guidelines for future prompt-based clinical NLP research, facilitating engagement by non-NLP experts in clinical NLP advancements. To the best of our knowledge, this is one of the first works on the empirical evaluation of different prompt engineering approaches for clinical NLP in this era of generative artificial intelligence, and we hope that it will inspire and inform future research in this area.

medical informatics
Clinical Prompt Learning With Frozen Language Models

Niall Taylor,Yi Zhang,Dan W. Joyce,Ziming Gao,Andrey Kormilitzin,Alejo Nevado-Holgado

DOI: https://doi.org/10.1109/tnnls.2023.3294633

IF: 14.255

2023-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:When the first transformer-based language models were published in the late 2010s, pretraining with general text and then fine-tuning the model on a task-specific dataset often achieved the state-of-the-art performance. However, more recent work suggests that for some tasks, directly prompting the pretrained model matches or surpasses fine-tuning in performance with few or no model parameter updates required. The use of prompts with language models for natural language processing (NLP) tasks is known as prompt learning. We investigated the viability of prompt learning on clinically meaningful decision tasks and directly compared this with more traditional fine-tuning methods. Results show that prompt learning methods were able to match or surpass the performance of traditional fine-tuning with up to 1000 times fewer trainable parameters, less training time, less training data, and lower computation resource requirements. We argue that these characteristics make prompt learning a very desirable alternative to traditional fine-tuning for clinical tasks, where the computational resources of public health providers are limited, and where data can often not be made available or not be used for fine-tuning due to patient privacy concerns. The complementary code to reproduce the experiments presented in this work can be found at https://github.com/NtaylorOX/Public_Clinical_Prompt.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
ChatGPT-HealthPrompt. Harnessing the Power of XAI in Prompt-Based Healthcare Decision Support using ChatGPT

Fatemeh Nazary,Yashar Deldjoo,Tommaso Di Noia

2023-08-18

Abstract:This study presents an innovative approach to the application of large language models (LLMs) in clinical decision-making, focusing on OpenAI's ChatGPT. Our approach introduces the use of contextual prompts-strategically designed to include task description, feature description, and crucially, integration of domain knowledge-for high-quality binary classification tasks even in data-scarce scenarios. The novelty of our work lies in the utilization of domain knowledge, obtained from high-performing interpretable ML models, and its seamless incorporation into prompt design. By viewing these ML models as medical experts, we extract key insights on feature importance to aid in decision-making processes. This interplay of domain knowledge and AI holds significant promise in creating a more insightful diagnostic tool. Additionally, our research explores the dynamics of zero-shot and few-shot prompt learning based on LLMs. By comparing the performance of OpenAI's ChatGPT with traditional supervised ML models in different data conditions, we aim to provide insights into the effectiveness of prompt engineering strategies under varied data availability. In essence, this paper bridges the gap between AI and healthcare, proposing a novel methodology for LLMs application in clinical decision support systems. It highlights the transformative potential of effective prompt design, domain knowledge integration, and flexible learning approaches in enhancing automated decision-making.

Artificial Intelligence,Computation and Language,Machine Learning
AdaPrompt: Adaptive Model Training for Prompt-based NLP

Yulong Chen,Yang Liu,Li Dong,Shuohang Wang,Chenguang Zhu,Michael Zeng,Yue Zhang

DOI: https://doi.org/10.48550/arXiv.2202.04824

2022-02-10

Computation and Language

Abstract:Prompt-based learning, with its capability to tackle zero-shot and few-shot NLP tasks, has gained much attention in community. The main idea is to bridge the gap between NLP downstream tasks and language modeling (LM), by mapping these tasks into natural language prompts, which are then filled by pre-trained language models (PLMs). However, for prompt learning, there are still two salient gaps between NLP tasks and pretraining. First, prompt information is not necessarily sufficiently present during LM pretraining. Second, task-specific data are not necessarily well represented during pretraining. We address these two issues by proposing AdaPrompt, adaptively retrieving external data for continual pretraining of PLMs by making use of both task and prompt characteristics. In addition, we make use of knowledge in Natural Language Inference models for deriving adaptive verbalizers. Experimental results on five NLP benchmarks show that AdaPrompt can improve over standard PLMs in few-shot settings. In addition, in zero-shot settings, our method outperforms standard prompt-based methods by up to 26.35\% relative error reduction.
Position-based Prompting for Health Outcome Generation

M. Abaho,D. Bollegala,P. Williamson,S. Dodd

DOI: https://doi.org/10.48550/arXiv.2204.03489

2022-03-30

Computation and Language

Abstract:Probing Pre-trained Language Models (PLMs) using prompts has indirectly implied that language models (LMs) can be treated as knowledge bases. To this end, this phenomena has been effective especially when these LMs are fine-tuned towards not just data of a specific domain, but also to the style or linguistic pattern of the prompts themselves. We observe that, satisfying a particular linguistic pattern in prompts is an unsustainable constraint that unnecessarily lengthens the probing task, especially because, they are often manually designed and the range of possible prompt template patterns can vary depending on the prompting objective and domain. We therefore explore an idea of using a position-attention mechanism to capture positional information of each word in a prompt relative to the mask to be filled, hence avoiding the need to re-construct prompts when the prompts linguistic pattern changes. Using our approach, we demonstrate the ability of eliciting answers to rare prompt templates (in a case study on health outcome generation) such as Postfix and Mixed patterns whose missing information is respectively at the start and in multiple random places of the prompt. More so, using various biomedical PLMs, our approach consistently outperforms a baseline in which the default mask language model (MLM) representation is used to predict masked tokens.
MED-Prompt: A novel prompt engineering framework for medicine prediction on free-text clinical notes

Awais Ahmed,Xiaoyang Zeng,Rui Xi,Mengshu Hou,Syed Attique Shah

DOI: https://doi.org/10.1016/j.jksuci.2024.101933

IF: 9.006

2024-01-22

Journal of King Saud University - Computer and Information Sciences

Abstract:Existing AI-based medicine prediction systems require substantial training time, computing resources, and extensive labeled data, yet they often lack scalability. To bridge these gaps, this study introduces a novel MED-Prompt framework that employs pretrained models such as BERT, BioBERT, and ClinicalBERT. The core of our framework lies in developing specialized prompts, which act as guiding instructions for the models during the prediction process. MED-Prompt develops prompts that help models interpret and extract medical information from clinical corpus. The clinical text was derived from the widely known MIMIC-III 1 dataset. The study performs a comparative analysis and evaluates the performance of Manual-Prompt and GPT-Prompts. Further, a fine-tuned approach is developed within MED-Prompt, leveraging transfer learning to achieve prompt-guided medicine predictions. The proposed method achieved a maximum F1-score of 96.8%, which is more than 40% F1-score higher than the pretrained model. In addition, the fine-tuned also showed an average of 2.38 times better processing performance. These results revealed that MED-Prompt is scalable regarding the number of training records and input prompts. These results not only demonstrate the proficiency and effectiveness of the framework but also significantly reduce computational requirements. This also indicates that the proposed approach has the potential to significantly improve patient care, reduce resource requirements, and increase the overall effectiveness of AI-driven medical prediction systems.

computer science, information systems
A Practical Survey on Zero-shot Prompt Design for In-context Learning

Yinheng Li

DOI: https://doi.org/10.26615/978-954-452-092-2_069

2023-09-23

Abstract:The remarkable advancements in large language models (LLMs) have brought about significant improvements in Natural Language Processing(NLP) tasks. This paper presents a comprehensive review of in-context learning techniques, focusing on different types of prompts, including discrete, continuous, few-shot, and zero-shot, and their impact on LLM performance. We explore various approaches to prompt design, such as manual design, optimization algorithms, and evaluation methods, to optimize LLM performance across diverse tasks. Our review covers key research studies in prompt engineering, discussing their methodologies and contributions to the field. We also delve into the challenges faced in evaluating prompt performance, given the absence of a single "best" prompt and the importance of considering multiple metrics. In conclusion, the paper highlights the critical role of prompt design in harnessing the full potential of LLMs and provides insights into the combination of manual design, optimization techniques, and rigorous evaluation for more effective and efficient use of LLMs in various NLP tasks.

Computation and Language,Artificial Intelligence,Emerging Technologies,Machine Learning
Exploring low-resource medical image classification with weakly supervised prompt learning

Fudan Zheng,Jindong Cao,Weijiang Yu,Zhiguang Chen,Nong Xiao,Yutong Lu

DOI: https://doi.org/10.1016/j.patcog.2024.110250

IF: 8

2024-01-08

Pattern Recognition

Abstract:Most advances in medical image recognition supporting clinical auxiliary diagnosis meet challenges due to the low-resource situation in the medical field, where annotations are highly expensive and professional. This low-resource problem can be alleviated by leveraging the transferable representations of large-scale pre-trained vision-language models like CLIP. After being pre-trained using large-scale unlabeled medical images and texts (such as medical reports), the vision-language models can learn transferable representations and support flexible downstream clinical tasks such as medical image classification via relevant medical text prompts. However, existing pre-trained vision-language models require domain experts (clinicians) to carefully design the medical text prompts based on different datasets when applied to specific medical image tasks, which is extremely time-consuming and greatly increases the burden on clinicians. To address this problem, we propose a weakly supervised prompt learning method MedPrompt for automatically generating medical prompts, which includes an unsupervised pre-trained vision-language model and a weakly supervised prompt learning model. The unsupervised pre-trained vision-language model adopts large-scale medical images and texts for pre-training, utilizing the natural correlation between medical images and corresponding medical texts without manual annotations. The weakly supervised prompt learning model only utilizes the classes of images in the dataset to guide the learning of the specific class vector in the prompt, while the learning of other context vectors in the prompt does not require any manual annotations for guidance. To the best of our knowledge, this is the first model to automatically generate medical prompts. With the assistance of these prompts, the pre-trained vision-language model can be freed from the strong expert dependency of manual annotation and manual prompt design, thus achieving end-to-end, low-cost medical image classification. Experimental results show that the model using our automatically generated prompts outperforms all its hand-crafted prompts counterparts in full-shot learning on all four datasets, and achieves superior accuracy on zero-shot image classification and few-shot learning in three of the four medical benchmark datasets and comparable accuracy in the remaining one. In addition, the proposed prompt generator is lightweight and therefore has the potential to be embedded into any network architecture.

computer science, artificial intelligence,engineering, electrical & electronic
Advanced PromptCBLUE Performance: A Novel Approach Leveraging Large Language Models

Hong‐Qing Ling,Congcong Ge,Jie Wang,Fuliang Quan,Jianping Zeng

2023-01-01

Abstract:PromptCBLUE is a Chinese medical NLP benchmark that converts sixteen different medical NLP tasks into text generation task through prompt learning. It requires completing all tasks with just one large language model backbone and keeping the fine-tuning parameters under 1% of the model size, posing huge challenges. To address this issue, we have proposed an innovative approach. Initially a two-stage fine-tuning process is employed. For intricate tasks, a task-decomposition prompting construction method is introduced, enhancing algorithmic efficacy. Moreover, the integration of focus loss is explored to further amplify algorithm performance. Using this approach, a score of 72.28 was achieved in the CCKS2023 PromptCBLUE general competition track. This research contributes new insights to the advancement of large language models in the medical field.
Prompt Engineering for Healthcare: Methodologies and Applications

Jiaqi Wang,Enze Shi,Sigang Yu,Zihao Wu,Chong Ma,Haixing Dai,Qiushi Yang,Yanqing Kang,Jinru Wu,Huawen Hu,Chenxi Yue,Haiyang Zhang,Yiheng Liu,Yi Pan,Zhengliang Liu,Lichao Sun,Xiang Li,Bao Ge,Xi Jiang,Dajiang Zhu,Yixuan Yuan,Dinggang Shen,Tianming Liu,Shu Zhang

DOI: https://doi.org/10.48550/arXiv.2304.14670

2024-03-23

Abstract:Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important in the healthcare domain. However, there is a lack of comprehensive reviews specifically focusing on prompt engineering in the medical field. This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. First, we will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications such as question-answering systems, text summarization, and machine translation. With the continuous improvement of general large language models, the importance of prompt engineering in the healthcare domain is becoming increasingly prominent. The aim of this article is to provide useful resources and bridges for healthcare natural language processing researchers to better explore the application of prompt engineering in this field. We hope that this review can provide new ideas and inspire for research and application in medical natural language processing.

Artificial Intelligence
Prompt Engineering Paradigms for Medical Applications: Scoping Review

Jamil Zaghir,Marco Naguib,Mina Bjelogrlic,Aurélie Névéol,Xavier Tannier,Christian Lovis

DOI: https://doi.org/10.2196/60501

IF: 7.076

2024-09-11

Journal of Medical Internet Research

Abstract:Background: Prompt engineering, focusing on crafting effective prompts to large language models (LLMs), has garnered attention for its capabilities at harnessing the potential of LLMs. This is even more crucial in the medical domain due to its specialized terminology and language technicity. Clinical natural language processing applications must navigate complex language and ensure privacy compliance. Prompt engineering offers a novel approach by designing tailored prompts to guide models in exploiting clinically relevant information from complex medical texts. Despite its promise, the efficacy of prompt engineering in the medical domain remains to be fully explored. Objective: The aim of the study is to review research efforts and technical approaches in prompt engineering for medical applications as well as provide an overview of opportunities and challenges for clinical practice. Methods: Databases indexing the fields of medicine, computer science, and medical informatics were queried in order to identify relevant published papers. Since prompt engineering is an emerging field, preprint databases were also considered. Multiple data were extracted, such as the prompt paradigm, the involved LLMs, the languages of the study, the domain of the topic, the baselines, and several learning, design, and architecture strategies specific to prompt engineering. We include studies that apply prompt engineering–based methods to the medical domain, published between 2022 and 2024, and covering multiple prompt paradigms such as prompt learning (PL), prompt tuning (PT), and prompt design (PD). Results: We included 114 recent prompt engineering studies. Among the 3 prompt paradigms, we have observed that PD is the most prevalent (78 papers). In 12 papers, PD, PL, and PT terms were used interchangeably. While ChatGPT is the most commonly used LLM, we have identified 7 studies using this LLM on a sensitive clinical data set. Chain-of-thought, present in 17 studies, emerges as the most frequent PD technique. While PL and PT papers typically provide a baseline for evaluating prompt-based approaches, 61% (48/78) of the PD studies do not report any nonprompt-related baseline. Finally, we individually examine each of the key prompt engineering–specific information reported across papers and find that many studies neglect to explicitly mention them, posing a challenge for advancing prompt engineering research. Conclusions: In addition to reporting on trends and the scientific landscape of prompt engineering, we provide reporting guidelines for future studies to help advance research in the medical field. We also disclose tables and figures summarizing medical prompt engineering papers available and hope that future contributions will leverage these existing works to better advance the field.

health care sciences & services,medical informatics
Pre-trained Language Models can be Fully Zero-Shot Learners

Xuandong Zhao,Siqi Ouyang,Zhiguo Yu,Ming Wu,Lei Li

DOI: https://doi.org/10.48550/arXiv.2212.06950

2022-12-14

Computation and Language

Abstract:How can we extend a pre-trained model to many language understanding tasks, without labeled or additional unlabeled data? Pre-trained language models (PLMs) have been effective for a wide range of NLP tasks. However, existing approaches either require fine-tuning on downstream labeled datasets or manually constructing proper prompts. In this paper, we propose nonparametric prompting PLM (NPPrompt) for fully zero-shot language understanding. Unlike previous methods, NPPrompt uses only pre-trained language models and does not require any labeled data or additional raw corpus for further fine-tuning, nor does it rely on humans to construct a comprehensive set of prompt label words. We evaluate NPPrompt against previous major few-shot and zero-shot learning methods on diverse NLP tasks: including text classification, text entailment, similar text retrieval, and paraphrasing. Experimental results demonstrate that our NPPrompt outperforms the previous best fully zero-shot method by big margins, with absolute gains of 12.8% in accuracy on text classification and 18.9% on the GLUE benchmark.
Zero-Label Prompt Selection

Chonghua Liao,Yanan Zheng,Zhilin Yang

DOI: https://doi.org/10.48550/arXiv.2211.04668

2022-11-09

Abstract:Natural language prompts have been shown to facilitate cross-task generalization for large language models. However, with no or limited labeled examples, the cross-task performance is highly sensitive to the choice of prompts, while selecting a high-performing prompt is challenging given the scarcity of labels. To address the issue, we propose a Zero-Label Prompt Selection (ZPS) method that selects prompts without any labeled data or gradient update. Specifically, given the candidate human-written prompts for a task, ZPS labels a set of unlabeled data with a prompt ensemble and uses the pseudo-labels for prompt selection. Experiments show that ZPS improves over prior methods by a sizeable margin in zero-label performance. We also extend ZPS to a few-shot setting and show its advantages over strong baselines such as prompt tuning and model tuning.

Computation and Language
Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering

Yan Hu,Qingyu Chen,Jingcheng Du,Xueqing Peng,Vipina Kuttichi Keloth,Xu Zuo,Yujia Zhou,Zehan Li,Xiaoqian Jiang,Zhiyong Lu,Kirk Roberts,Hua Xu

DOI: https://doi.org/10.48550/arXiv.2303.16416

2024-01-25

Abstract:Objective: This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks and proposes task-specific prompts to improve their performance. Materials and Methods: We evaluated these models on two clinical NER tasks: (1) to extract medical problems, treatments, and tests from clinical notes in the MTSamples corpus, following the 2010 i2b2 concept extraction shared task, and (2) identifying nervous system disorder-related adverse events from safety reports in the vaccine adverse event reporting system (VAERS). To improve the GPT models' performance, we developed a clinical task-specific prompt framework that includes (1) baseline prompts with task description and format specification, (2) annotation guideline-based prompts, (3) error analysis-based instructions, and (4) annotated samples for few-shot learning. We assessed each prompt's effectiveness and compared the models to BioClinicalBERT. Results: Using baseline prompts, GPT-3.5 and GPT-4 achieved relaxed F1 scores of 0.634, 0.804 for MTSamples, and 0.301, 0.593 for VAERS. Additional prompt components consistently improved model performance. When all four components were used, GPT-3.5 and GPT-4 achieved relaxed F1 socres of 0.794, 0.861 for MTSamples and 0.676, 0.736 for VAERS, demonstrating the effectiveness of our prompt framework. Although these results trail BioClinicalBERT (F1 of 0.901 for the MTSamples dataset and 0.802 for the VAERS), it is very promising considering few training samples are needed. Conclusion: While direct application of GPT models to clinical NER tasks falls short of optimal performance, our task-specific prompt framework, incorporating medical knowledge and training samples, significantly enhances GPT models' feasibility for potential clinical applications.

Computation and Language
Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data

Yinghao Zhu,Zixiang Wang,Junyi Gao,Yuning Tong,Jingkun An,Weibin Liao,Ewen M. Harrison,Liantao Ma,Chengwei Pan

2024-02-11

Abstract:The inherent complexity of structured longitudinal Electronic Health Records (EHR) data poses a significant challenge when integrated with Large Language Models (LLMs), which are traditionally tailored for natural language processing. Motivated by the urgent need for swift decision-making during new disease outbreaks, where traditional predictive models often fail due to a lack of historical data, this research investigates the adaptability of LLMs, like GPT-4, to EHR data. We particularly focus on their zero-shot capabilities, which enable them to make predictions in scenarios in which they haven't been explicitly trained. In response to the longitudinal, sparse, and knowledge-infused nature of EHR data, our prompting approach involves taking into account specific EHR characteristics such as units and reference ranges, and employing an in-context learning strategy that aligns with clinical contexts. Our comprehensive experiments on the MIMIC-IV and TJH datasets demonstrate that with our elaborately designed prompting framework, LLMs can improve prediction performance in key tasks such as mortality, length-of-stay, and 30-day readmission by about 35\%, surpassing ML models in few-shot settings. Our research underscores the potential of LLMs in enhancing clinical decision-making, especially in urgent healthcare situations like the outbreak of emerging diseases with no labeled data. The code is publicly available at

Computation and Language,Artificial Intelligence,Machine Learning
Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records

Erlend Frayling,Jake Lever,Graham McDonald

DOI: https://doi.org/10.48550/arXiv.2403.08664

2024-03-14

Abstract:The challenge of accessing historical patient data for clinical research, while adhering to privacy regulations, is a significant obstacle in medical science. An innovative approach to circumvent this issue involves utilising synthetic medical records that mirror real patient data without compromising individual privacy. The creation of these synthetic datasets, particularly without using actual patient data to train Large Language Models (LLMs), presents a novel solution as gaining access to sensitive patient information to train models is also a challenge. This study assesses the capability of the Llama 2 LLM to create synthetic medical records that accurately reflect real patient information, employing zero-shot and few-shot prompting strategies for comparison against fine-tuned methodologies that do require sensitive patient data during training. We focus on generating synthetic narratives for the History of Present Illness section, utilising data from the MIMIC-IV dataset for comparison. In this work introduce a novel prompting technique that leverages a chain-of-thought approach, enhancing the model's ability to generate more accurate and contextually relevant medical narratives without prior fine-tuning. Our findings suggest that this chain-of-thought prompted approach allows the zero-shot model to achieve results on par with those of fine-tuned models, based on Rouge metrics evaluation.

Computation and Language,Machine Learning
Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models

Ran Xu,Hejie Cui,Yue Yu,Xuan Kan,Wenqi Shi,Yuchen Zhuang,Wei Jin,Joyce Ho,Carl Yang

2023-11-01

Abstract:Clinical natural language processing requires methods that can address domain-specific challenges, such as complex medical terminology and clinical contexts. Recently, large language models (LLMs) have shown promise in this domain. Yet, their direct deployment can lead to privacy issues and are constrained by resources. To address this challenge, we delve into synthetic clinical text generation using LLMs for clinical NLP tasks. We propose an innovative, resource-efficient approach, ClinGen, which infuses knowledge into the process. Our model involves clinical knowledge extraction and context-informed LLM prompting. Both clinical topics and writing styles are drawn from external domain-specific knowledge graphs and LLMs to guide data generation. Our extensive empirical study across 7 clinical NLP tasks and 16 datasets reveals that ClinGen consistently enhances performance across various tasks, effectively aligning the distribution of real datasets and significantly enriching the diversity of generated training instances. We will publish our code and all the generated data in \url{<a class="link-external link-https" href="https://github.com/ritaranx/ClinGen" rel="external noopener nofollow">this https URL</a>}.

Computation and Language,Artificial Intelligence,Machine Learning,Quantitative Methods
Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting

Phillip Richter-Pechanski,Philipp Wiesenbach,Dominic M. Schwab,Christina Kiriakou,Nicolas Geis,Christoph Dieterich,Anette Frank

2024-08-13

Abstract:Automatic extraction of medical information from clinical documents poses several challenges: high costs of required clinical expertise, limited interpretability of model predictions, restricted computational resources and privacy regulations. Recent advances in domain-adaptation and prompting methods showed promising results with minimal training data using lightweight masked language models, which are suited for well-established interpretability methods. We are first to present a systematic evaluation of these methods in a low-resource setting, by performing multi-class section classification on German doctor's letters. We conduct extensive class-wise evaluations supported by Shapley values, to validate the quality of our small training data set and to ensure the interpretability of model predictions. We demonstrate that a lightweight, domain-adapted pretrained model, prompted with just 20 shots, outperforms a traditional classification model by 30.5% accuracy. Our results serve as a process-oriented guideline for clinical information extraction projects working with low-resource.

Computation and Language,Artificial Intelligence,Machine Learning
XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization

Yequan Bie,Luyang Luo,Zhixuan Chen,Hao Chen

2024-03-14

Abstract:Utilizing potent representations of the large vision-language models (VLMs) to accomplish various downstream tasks has attracted increasing attention. Within this research field, soft prompt learning has become a representative approach for efficiently adapting VLMs such as CLIP, to tasks like image classification. However, most existing prompt learning methods learn text tokens that are unexplainable, which cannot satisfy the stringent interpretability requirements of Explainable Artificial Intelligence (XAI) in high-stakes scenarios like healthcare. To address this issue, we propose a novel explainable prompt learning framework that leverages medical knowledge by aligning the semantics of images, learnable prompts, and clinical concept-driven prompts at multiple granularities. Moreover, our framework addresses the lack of valuable concept annotations by eliciting knowledge from large language models and offers both visual and textual explanations for the prompts. Extensive experiments and explainability analyses conducted on various datasets, with and without concept labels, demonstrate that our method simultaneously achieves superior diagnostic performance, flexibility, and interpretability, shedding light on the effectiveness of foundation models in facilitating XAI. The code will be made publically available.

Computer Vision and Pattern Recognition,Artificial Intelligence

Inhibitors of prostaglandin, prostacyclin, and thromboxane synthesis.

An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study

Clinical Prompt Learning With Frozen Language Models

ChatGPT-HealthPrompt. Harnessing the Power of XAI in Prompt-Based Healthcare Decision Support using ChatGPT

AdaPrompt: Adaptive Model Training for Prompt-based NLP

Position-based Prompting for Health Outcome Generation

MED-Prompt: A novel prompt engineering framework for medicine prediction on free-text clinical notes

A Practical Survey on Zero-shot Prompt Design for In-context Learning

Exploring low-resource medical image classification with weakly supervised prompt learning

Advanced PromptCBLUE Performance: A Novel Approach Leveraging Large Language Models

Prompt Engineering for Healthcare: Methodologies and Applications

Prompt Engineering Paradigms for Medical Applications: Scoping Review

Pre-trained Language Models can be Fully Zero-Shot Learners

Zero-Label Prompt Selection

Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering

Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data

Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records

Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models

Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting

XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization