Abstract:Background: Mining the vast pool of biomedical literature to extract accurate responses and relevant references is challenging due to the domain's interdisciplinary nature, specialized jargon, and continuous evolution. Early natural language processing (NLP) approaches often led to incorrect answers as they failed to comprehend the nuances of natural language. However, transformer models have significantly advanced the field by enabling the creation of large language models (LLMs), enhancing question-answering (QA) tasks. Despite these advances, current LLM-based solutions for specialized domains like biology and biomedicine still struggle to generate up-to-date responses while avoiding "hallucination" or generating plausible but factually incorrect responses. Results: Our work focuses on enhancing prompts using a retrieval-augmented architecture to guide LLMs in generating meaningful responses for biomedical QA tasks. We evaluated two approaches: one relying on text embedding and vector similarity in a high-dimensional space, and our proposed method, which uses explicit signals in user queries to extract meaningful contexts. For robust evaluation, we tested these methods on 50 specific and challenging questions from diverse biomedical topics, comparing their performance against a baseline model, BM25. Retrieval performance of our method was significantly better than others, achieving a median Precision@10 of 0.95, which indicates the fraction of the top 10 retrieved chunks that are relevant. We used GPT-4, OpenAI's most advanced LLM to maximize the answer quality and manually accessed LLM-generated responses. Our method achieved a median answer quality score of 2.5, surpassing both the baseline model and the text embedding-based approach. We developed a QA bot, WeiseEule ( https://github.com/wasimaftab/WeiseEule-LocalHost ), which utilizes these methods for comparative analysis and also offers advanced features for review writing and identifying relevant articles for citation. Conclusions: Our findings highlight the importance of prompt enhancement methods that utilize explicit signals in user queries over traditional text embedding-based approaches to improve LLM-generated responses for specialized queries in specialized domains such as biology and biomedicine. By providing users complete control over the information fed into the LLM, our approach addresses some of the major drawbacks of existing web-based chatbots and LLM-based QA systems, including hallucinations and the generation of irrelevant or outdated responses.

CBAG: Conditional biomedical abstract generation

Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers

NeuroSym-BioCAT: Leveraging Neuro-Symbolic Methods for Biomedical Scholarly Document Categorization and Question Answering

Biomedical text summarization using Conditional Generative Adversarial Network(CGAN)

Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts

Controllable Contrastive Generation for Multilingual Biomedical Entity Linking

Leveraging Large Language Models for Clinical Abbreviation Disambiguation

Unified Representation of Genomic and Biomedical Concepts through Multi-Task, Multi-Source Contrastive Learning

Transformers and the Representation of Biomedical Background Knowledge

A concept-driven biomedical knowledge extraction and visualization framework for conceptualization of text corpora

AI for Biomedicine in the Era of Large Language Models

Generating Biomedical Hypothesis with Spatiotemporal Transformers

Learning Disentangled Representations of Texts with Application to Biomedical Abstracts

Keyphrase Generation Beyond the Boundaries of Title and Abstract

MOLIERE: Automatic Biomedical Hypothesis Generation System

Continual knowledge infusion into pre-trained biomedical language models

Explainable Biomedical Hypothesis Generation via Retrieval Augmented Generation enabled Large Language Models

Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy

A BERT-Based Generation Model to Transform Medical Texts to SQL Queries for Electronic Medical Records: Model Development and Validation

SciFive: a text-to-text transformer model for biomedical literature