Inhibition of brain sodium-potassium ATPase in uremic rats.

L. Minkoff,G. Gaertner,M. Darab,C. Mercier,Levin Ml

1972-07-01

Journal of Laboratory and Clinical Medicine

Abstract:

What problem does this paper attempt to address?

Language Enhanced Model for Eye (LEME): An Open-Source Ophthalmology-Specific Large Language Model

Aidan Gilson,Xuguang Ai,Qianqian Xie,Sahana Srinivasan,Krithi Pushpanathan,Maxwell B. Singer,Jimin Huang,Hyunjae Kim,Erping Long,Peixing Wan,Luciano V. Del Priore,Lucila Ohno-Machado,Hua Xu,Dianbo Liu,Ron A. Adelman,Yih-Chung Tham,Qingyu Chen

2024-10-01

Abstract:Large Language Models (LLMs) are poised to revolutionize healthcare. Ophthalmology-specific LLMs remain scarce and underexplored. We introduced an open-source, specialized LLM for ophthalmology, termed Language Enhanced Model for Eye (LEME). LEME was initially pre-trained on the Llama2 70B framework and further fine-tuned with a corpus of ~127,000 non-copyrighted training instances curated from ophthalmology-specific case reports, abstracts, and open-source study materials. We benchmarked LEME against eight other LLMs, namely, GPT-3.5, GPT-4, three Llama2 models (7B, 13B, 70B), PMC-LLAMA 13B, Meditron 70B, and EYE-Llama (another ophthalmology-specific LLM). Evaluations included four internal validation tasks: abstract completion, fill-in-the-blank, multiple-choice questions (MCQ), and short-answer QA. External validation tasks encompassed long-form QA, MCQ, patient EHR summarization, and clinical QA. Evaluation metrics included Rouge-L scores, accuracy, and expert evaluation of correctness, completeness, and readability. In internal validations, LEME consistently outperformed its counterparts, achieving Rouge-L scores of 0.20 in abstract completion (all p<0.05), 0.82 in fill-in-the-blank (all p<0.0001), and 0.22 in short-answer QA (all p<0.0001, except versus GPT-4). In external validations, LEME excelled in long-form QA with a Rouge-L of 0.19 (all p<0.0001), ranked second in MCQ accuracy (0.68; all p<0.0001), and scored highest in EHR summarization and clinical QA (ranging from 4.24 to 4.83 out of 5 for correctness, completeness, and readability). LEME's emphasis on robust fine-tuning and the use of non-copyrighted data represents a breakthrough in open-source ophthalmology-specific LLMs, offering the potential to revolutionize execution of clinical tasks while democratizing research collaboration.

Computation and Language
Ophtha-LLaMA2: A Large Language Model for Ophthalmology

Huan Zhao,Qian Ling,Yi Pan,Tianyang Zhong,Jin-Yu Hu,Junjie Yao,Fengqian Xiao,Zhenxiang Xiao,Yutong Zhang,San-Hua Xu,Shi-Nan Wu,Min Kang,Zihao Wu,Zhengliang Liu,Xi Jiang,Tianming Liu,Yi Shao

DOI: https://doi.org/10.48550/arXiv.2312.04906

2023-12-08

Computation and Language

Abstract:In recent years, pre-trained large language models (LLMs) have achieved tremendous success in the field of Natural Language Processing (NLP). Prior studies have primarily focused on general and generic domains, with relatively less research on specialized LLMs in the medical field. The specialization and high accuracy requirements for diagnosis in the medical field, as well as the challenges in collecting large-scale data, have constrained the application and development of LLMs in medical scenarios. In the field of ophthalmology, clinical diagnosis mainly relies on doctors' interpretation of reports and making diagnostic decisions. In order to take advantage of LLMs to provide decision support for doctors, we collected three modalities of ophthalmic report data and fine-tuned the LLaMA2 model, successfully constructing an LLM termed the "Ophtha-LLaMA2" specifically tailored for ophthalmic disease diagnosis. Inference test results show that even with a smaller fine-tuning dataset, Ophtha-LLaMA2 performs significantly better in ophthalmic diagnosis compared to other LLMs. It demonstrates that the Ophtha-LLaMA2 exhibits satisfying accuracy and efficiency in ophthalmic disease diagnosis, making it a valuable tool for ophthalmologists to provide improved diagnostic support for patients. This research provides a useful reference for the application of LLMs in the field of ophthalmology, while showcasing the immense potential and prospects in this domain.
Medical education with large language models in ophthalmology: custom instructions and enhanced retrieval capabilities

Mertcan Sevgi,Fares Antaki,Pearse A Keane

DOI: https://doi.org/10.1136/bjo-2023-325046

2024-05-07

Abstract:Foundation models are the next generation of artificial intelligence that has the potential to provide novel use cases for healthcare. Large language models (LLMs), a type of foundation model, are capable of language comprehension and the ability to generate human-like text. Researchers and developers have been tuning LLMs to optimise their performance in specific tasks, such as medical challenge problems. Until recently, tuning required technical programming expertise, but the release of custom generative pre-trained transformers (GPTs) by OpenAI has allowed users to tune their own GPTs with natural language. This has the potential to democratise access to high-quality bespoke LLMs globally. In this review, we provide an overview of LLMs, how they are tuned and how custom GPTs work. We provide three use cases of custom GPTs in ophthalmology to demonstrate the versatility and effectiveness of these tools. First, we present 'EyeTeacher', an educational aid that generates questions from clinical guidelines to facilitate learning. Second, we built 'EyeAssistant', a clinical support tool that is tuned with clinical guidelines to respond to various physician queries. Lastly, we design 'The GPT for GA', which offers clinicians a comprehensive summary of emerging management strategies for geographic atrophy by analysing peer-reviewed documents. The review underscores the significance of custom instructions and information retrieval in tuning GPTs for specific tasks in ophthalmology. We also discuss the evaluation of LLM responses and address critical aspects such as privacy and accountability in their clinical application. Finally, we discuss their potential in ophthalmic education and clinical practice.
Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

Lifeng Zhu,Jingping Liu,Jiacheng Wang,Weiyan Zhang,Sihang Jiang,Hai Yang,Chao Wang,Qi Ye,Tong Ruan,Xinkai Rui,Huajun Chen

DOI: https://doi.org/10.21203/rs.3.rs-5285540/v1

2024-01-01

Abstract:Electronic medical records (EMRs) are essential in clinical practice. Although current medical large language models (LLMs) excel in tasks like US Medical Licensing Examination, they struggle with real-world clinical applications due to insufficient large-scale EMR data in their training, hindering their clinical expertise. To address this limitation, we proposed EMR-LLM, an LLM for clinical practice using EMRs. Firstly, we continually pre-trained a general LLM on medical corpora to enhance its domain knowledge. Then, we designed three categories of instruction tasks using EMRs: structure understanding, numerical understanding, and downstream tasks. Finally, we introduced an ability-boosting instruction-tuning method, which mimics human learning, progressing from simple to complex tasks while introducing a data replay strategy to retain learned knowledge. Experimental results demonstrated that EMR-LLM outperformed strong competitors on six EMR tasks, nine medical benchmarks, and three open-domain benchmarks. Moreover, in discharge summary generation, EMR-LLM achieved performance levels close to those of expert clinicians.
EyeGPT: Ophthalmic Assistant with Large Language Models

Xiaolan Chen,Ziwei Zhao,Weiyi Zhang,Pusheng Xu,Le Gao,Mingpu Xu,Yue Wu,Yinwen Li,Danli Shi,Mingguang He

2024-02-29

Abstract:Artificial intelligence (AI) has gained significant attention in healthcare consultation due to its potential to improve clinical workflow and enhance medical communication. However, owing to the complex nature of medical information, large language models (LLM) trained with general world knowledge might not possess the capability to tackle medical-related tasks at an expert level. Here, we introduce EyeGPT, a specialized LLM designed specifically for ophthalmology, using three optimization strategies including role-playing, finetuning, and retrieval-augmented generation. In particular, we proposed a comprehensive evaluation framework that encompasses a diverse dataset, covering various subspecialties of ophthalmology, different users, and diverse inquiry intents. Moreover, we considered multiple evaluation metrics, including accuracy, understandability, trustworthiness, empathy, and the proportion of hallucinations. By assessing the performance of different EyeGPT variants, we identify the most effective one, which exhibits comparable levels of understandability, trustworthiness, and empathy to human ophthalmologists (all Ps>0.05). Overall, ur study provides valuable insights for future research, facilitating comprehensive comparisons and evaluations of different strategies for developing specialized LLMs in ophthalmology. The potential benefits include enhancing the patient experience in eye care and optimizing ophthalmologists' services.

Computation and Language,Artificial Intelligence
OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue

Weihao Gao,Zhuo Deng,Zhiyuan Niu,Fuju Rong,Chucheng Chen,Zheng Gong,Wenze Zhang,Daimin Xiao,Fang Li,Zhenjie Cao,Zhaoyi Ma,Wenbin Wei,Lan Ma

2023-06-22

Abstract:Large multimodal language models (LMMs) have achieved significant success in general domains. However, due to the significant differences between medical images and text and general web content, the performance of LMMs in medical scenarios is limited. In ophthalmology, clinical diagnosis relies on multiple modalities of medical images, but unfortunately, multimodal ophthalmic large language models have not been explored to date. In this paper, we study and construct an ophthalmic large multimodal model. Firstly, we use fundus images as an entry point to build a disease assessment and diagnosis pipeline to achieve common ophthalmic disease diagnosis and lesion segmentation. Then, we establish a new ophthalmic multimodal instruction-following and dialogue fine-tuning dataset based on disease-related knowledge data and publicly available real-world medical dialogue. We introduce visual ability into the large language model to complete the ophthalmic large language and vision assistant (OphGLM). Our experimental results demonstrate that the OphGLM model performs exceptionally well, and it has the potential to revolutionize clinical applications in ophthalmology. The dataset, code, and models will be made publicly available at <a class="link-external link-https" href="https://github.com/ML-AILab/OphGLM" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
A case of IgE multiple myeloma

L. Lloyd,S. Klingberg,M. Kende,J. F. Howell,A. Clague

DOI: https://doi.org/10.1080/0031302021000062424

2003-02-01

Abstract:
Utilizing Large Language Models in Ophthalmology: The Current Landscape and Challenges

Peranut Chotcomwongse,Paisan Ruamviboonsuk,Andrzej Grzybowski

DOI: https://doi.org/10.1007/s40123-024-01018-6

2024-08-24

Abstract:A large language model (LLM) is an artificial intelligence (AI) model that uses natural language processing (NLP) to understand, interpret, and generate human-like language responses from unstructured text input. Its real-time response capabilities and eloquent dialogue enhance the interactive user experience in human-AI communication like never before. By gathering several sources on the internet, LLM chatbots can interact and respond to a wide range of queries, including problem solving, text summarization, and creating informative notes. Since ophthalmology is one of the medical fields integrating image analysis, telemedicine, AI, and other technologies, LLMs are likely to play an important role in eye care in the near future. This review summarizes the performance and potential applicability of LLMs in ophthalmology according to currently available publications.
Large Language Models in Ophthalmology: Potential and Pitfalls

Antonio Yaghy,Maria Yaghy,Jerry A. Shields,Carol L. Shields

DOI: https://doi.org/10.1080/08820538.2023.2300808

2024-01-07

Seminars in Ophthalmology

Abstract:Large language models (LLMs) show great promise in assisting clinicians in general, and ophthalmology in particular, through knowledge synthesis, decision support, accelerating research, enhancing education, and improving patient interactions. Specifically, LLMs can rapidly summarize the latest literature to keep clinicians up-to-date. They can also analyze patient data to highlight crucial insights and recommend appropriate tests or referrals. LLMs can automate tedious research tasks like data cleaning and literature reviews. As AI tutors, LLMs can fill knowledge gaps and assess competency in trainees. As chatbots, they can provide empathetic, personalized responses to patient inquiries and improve satisfaction. The visual capabilities of LLMs like GPT-4 allow assisting the visually impaired by describing environments. However, there are significant ethical, technical, and legal challenges around the use of LLMs that should be addressed regarding privacy, fairness, robustness, attribution, and regulation. Ongoing oversight and refinement of models is critical to realize benefits while minimizing risks and upholding responsible AI principles. If carefully implemented, LLMs hold immense potential to push the boundaries of care, discovery, and quality of life for ophthalmology patients.

ophthalmology
OphGLM: An ophthalmology large language-and-vision assistant

Zhuo Deng,Weihao Gao,Chucheng Chen,Zhiyuan Niu,Zheng Gong,Ruiheng Zhang,Zhenjie Cao,Fang Li,Zhaoyi Ma,Wenbin Wei,Lan Ma

DOI: https://doi.org/10.1016/j.artmed.2024.103001

IF: 7.011

2024-10-24

Artificial Intelligence in Medicine

Abstract:Vision computer-aided diagnostic methods have been used in early ophthalmic disease screening and diagnosis. However, the limited output formats of these methods lead to poor human–computer interaction and low clinical applicability value. Thus, ophthalmic visual question answering is worth studying. Unfortunately, no practical solutions exist before Large Language Models(LLMs). In this paper, we investigate the ophthalmic visual diagnostic interaction problem. We construct an ophthalmology large language-and-vision assistant, OphGLM, consisting of an image encoder, a text encoder, a fusion module, and an LLM module. We establish a new Chinese ophthalmic fine-tuning dataset, FundusTuning-CN, including the fundus instruction and conversation sets. Based on FundusTuning-CN, we establish a novel LLM-tuning strategy to introduce visual model understanding and ophthalmic knowledge into LLMs at a low cost and high efficiency. Leveraging the pre-training of the image encoder, OphGLM demonstrates strong visual understanding and surpasses open-source visual language models in common fundus disease classification tasks. The FundusTuning-CN enables OphGLM to surpass open-source medical LLMs in both ophthalmic knowledge and interactive capabilities. Our proposed OphGLM has the potential to revolutionize clinical applications in ophthalmology. The dataset, code, and models will be publicly available at https://github.com/ML-AILab/OphGLM .

engineering, biomedical,computer science, artificial intelligence,medical informatics
Medical, moral and legal aspects of renal replacement therapy.

A. Siemsen,T. Beauchamp,J. Robertson

1977-12-01

Abstract:
LlamaCare: A Large Medical Language Model for Enhancing Healthcare Knowledge Sharing

Maojun Sun

2024-06-05

Abstract:Large language models (LLMs) have shown amazing capabilities in knowledge memorization and the present. However, when it comes to domain-specific knowledge and downstream tasks like medical, general LLMs are often unable to give precise answers. In addition, when people want LLMs to answer classification questions, they usually go through instruction tuning first. However, LLMs do not always give a direct index of the categorization after instruction tuning. In this paper, we proposed LlamaCare, a fine-tuned medical language model, and Extended Classification Integration(ECI), a module to handle classification problems of LLMs. Our contributions are : (i) We fine-tuned a large language model of medical knowledge with very low carbon emissions and achieved similar performance with ChatGPT by a 24G GPU. (ii) We solved the problem of redundant categorical answers and improved the performance of LLMs by proposing a new module called Extended Classification Integration. (iii) We released our processed data for one-shot and few-shot training for some benchmarks such as PubMedQA and USMLE 1-3 step. Our method achieves a close performance comparable to some state-of-the-art models with the same quantity of parameters on benchmarks, while being more environmentally friendly by using less GPU computation time. Our models, codes, and datasets can be found at \url{<a class="link-external link-https" href="https://github.com/Stephen-SMJ/LLamaCare" rel="external noopener nofollow">this https URL</a>}.

Computation and Language,Artificial Intelligence
Development and evaluation of a large language model of ophthalmology in Chinese

Ce Zheng,Hongfei Ye,Jinming Guo,Junrui Yang,Ping Fei,Yuanzhi Yuan,Danqing Huang,Yuqiang Huang,Jie Peng,Xiaoling Xie,Meng Xie,Peiquan Zhao,Li Chen,Mingzhi Zhang

DOI: https://doi.org/10.1136/bjo-2023-324526

2024-07-17

Abstract:Background: Large language models (LLMs), such as ChatGPT, have considerable implications for various medical applications. However, ChatGPT's training primarily draws from English-centric internet data and is not tailored explicitly to the medical domain. Thus, an ophthalmic LLM in Chinese is clinically essential for both healthcare providers and patients in mainland China. Methods: We developed an LLM of ophthalmology (MOPH) using Chinese corpora and evaluated its performance in three clinical scenarios: ophthalmic board exams in Chinese, answering evidence-based medicine-oriented ophthalmic questions and diagnostic accuracy for clinical vignettes. Additionally, we compared MOPH's performance to that of human doctors. Results: In the ophthalmic exam, MOPH's average score closely aligned with the mean score of trainees (64.7 (range 62-68) vs 66.2 (range 50-92), p=0.817), but achieving a score above 60 in all seven mock exams. In answering ophthalmic questions, MOPH demonstrated an adherence of 83.3% (25/30) of responses following Chinese guidelines (Likert scale 4-5). Only 6.7% (2/30, Likert scale 1-2) and 10% (3/30, Likert scale 3) of responses were rated as 'poor or very poor' or 'potentially misinterpretable inaccuracies' by reviewers. In diagnostic accuracy, although the rate of correct diagnosis by ophthalmologists was superior to that by MOPH (96.1% vs 81.1%, p>0.05), the difference was not statistically significant. Conclusion: This study demonstrated the promising performance of MOPH, a Chinese-specific ophthalmic LLM, in diverse clinical scenarios. MOPH has potential real-world applications in Chinese-language ophthalmology settings.
Exploring large language model for next generation of artificial intelligence in ophthalmology

Kai Jin,Lu Yuan,Hongkang Wu,Andrzej Grzybowski,Juan Ye

DOI: https://doi.org/10.3389/fmed.2023.1291404

IF: 3.9

2023-11-23

Frontiers in Medicine

Abstract:In recent years, ophthalmology has advanced significantly, thanks to rapid progress in artificial intelligence (AI) technologies. Large language models (LLMs) like ChatGPT have emerged as powerful tools for natural language processing. This paper finally includes 108 studies, and explores LLMs’ potential in the next generation of AI in ophthalmology. The results encompass a diverse range of studies in the field of ophthalmology, highlighting the versatile applications of LLMs. Subfields encompass general ophthalmology, retinal diseases, anterior segment diseases, glaucoma, and ophthalmic plastics. Results show LLMs’ competence in generating informative and contextually relevant responses, potentially reducing diagnostic errors and improving patient outcomes. Overall, this study highlights LLMs’ promising role in shaping AI’s future in ophthalmology. By leveraging AI, ophthalmologists can access a wealth of information, enhance diagnostic accuracy, and provide better patient care. Despite challenges, continued AI advancements and ongoing research will pave the way for the next generation of AI-assisted ophthalmic practices.

medicine, general & internal
From Text to Tables: A Local Privacy Preserving Large Language Model for Structured Information Retrieval from Medical Documents

Isabella Catharina Wiest,Dyke Ferber,Jiefu Zhu,Marko Van Treeck,Sonja Katharina Meyer,Radhika Juglan,Zunamys I. Carrero,Daniel Paech,Jens Kleesiek,Matthias P. Ebert,Daniel Truhn,Jakob Nikolas Kather

DOI: https://doi.org/10.1101/2023.12.07.23299648

2023-12-10

MedRxiv

Abstract:Background and Aims Most clinical information is encoded as text, but extracting quantitative information from text is challenging. Large Language Models (LLMs) have emerged as powerful tools for natural language processing and can parse clinical text. However, many LLMs including ChatGPT reside in remote data centers, which disqualifies them from processing personal healthcare data. We present an open-source pipeline using the local LLM 'Llama 2' for extracting quantitative information from clinical text and evaluate its use to detect clinical features of decompensated liver cirrhosis. Methods We tasked the LLM to identify five key clinical features of decompensated liver cirrhosis in a zero- and one-shot way without any model training. Our specific objective was to identify abdominal pain, shortness of breath, confusion, liver cirrhosis, and ascites from 500 patient medical histories from the MIMIC IV dataset. We compared LLMs with three different sizes and a variety of pre-specified prompt engineering approaches. Model predictions were compared against the ground truth provided by the consent of three blinded medical experts. Results Our open-source pipeline yielded in highly accurate extraction of quantitative features from medical free text. Clinical features which were explicitly mentioned in the source text, such as liver cirrhosis and ascites, were detected with a sensitivity of 100% and 95% and a specificity of 96% and 95%, respectively from the 70 billion parameter model. Other clinical features, which are often paraphrased in a variety of ways, such as the presence of confusion, were detected only with a sensitivity of 76% and a specificity of 94%. Abdominal pain was detected with a sensitivity of 84% and a specificity of 97%. Shortness of breath was detected with a sensitivity of 87% and a specificity of 96%. The larger version of Llama 2 with 70b parameters outperformed the smaller version with 7b parameters in all tasks. Prompt engineering improved zero-shot performance, particularly for smaller model sizes. Conclusion Our study successfully demonstrates the capability of using locally deployed LLMs to extract clinical information from free text. The hardware requirements are so low that not only on-premise, but also point-of-care deployment of LLMs are possible.
Me LLaMA: Foundation Large Language Models for Medical Applications

Qianqian Xie,Qingyu Chen,Aokun Chen,Cheng Peng,Yan Hu,Fongci Lin,Xueqing Peng,Jimin Huang,Jeffrey Zhang,Vipina Keloth,Xinyu Zhou,Huan He,Lucila Ohno-Machado,Yonghui Wu,Hua Xu,Jiang Bian

2024-04-12

Abstract:Recent advancements in large language models (LLMs) such as ChatGPT and LLaMA have hinted at their potential to revolutionize medical applications, yet their application in clinical settings often reveals limitations due to a lack of specialized training on medical-specific data. In response to this challenge, this study introduces Me-LLaMA, a novel medical LLM family that includes foundation models - Me-LLaMA 13/70B, along with their chat-enhanced versions - Me-LLaMA 13/70B-chat, developed through continual pre-training and instruction tuning of LLaMA2 using large medical datasets. Our methodology leverages a comprehensive domain-specific data suite, including a large-scale, continual pre-training dataset with 129B tokens, an instruction tuning dataset with 214k samples, and a new medical evaluation benchmark (MIBE) across six critical medical tasks with 12 datasets. Our extensive evaluation using the MIBE shows that Me-LLaMA models achieve overall better performance than existing open-source medical LLMs in zero-shot, few-shot and supervised learning abilities. With task-specific instruction tuning, Me-LLaMA models outperform ChatGPT on 7 out of 8 datasets and GPT-4 on 5 out of 8 datasets. In addition, we investigated the catastrophic forgetting problem, and our results show that Me-LLaMA models outperform other open-source medical LLMs in mitigating this issue. Me-LLaMA is one of the largest open-source medical foundation LLMs that use both biomedical and clinical data. It exhibits superior performance across both general and medical tasks compared to other open-source medical LLMs, rendering it an attractive choice for medical AI applications. We release our models, datasets, and evaluation scripts at:

Artificial Intelligence
Review of emerging trends and projection of future developments in large language models research in ophthalmology

Matthew Wong,Zhi Wei Lim,Krithi Pushpanathan,Carol Y Cheung,Ya Xing Wang,David Chen,Yih Chung Tham

DOI: https://doi.org/10.1136/bjo-2023-324734

2024-09-20

Abstract:Background: Large language models (LLMs) are fast emerging as potent tools in healthcare, including ophthalmology. This systematic review offers a twofold contribution: it summarises current trends in ophthalmology-related LLM research and projects future directions for this burgeoning field. Methods: We systematically searched across various databases (PubMed, Europe PMC, Scopus and Web of Science) for articles related to LLM use in ophthalmology, published between 1 January 2022 and 31 July 2023. Selected articles were summarised, and categorised by type (editorial, commentary, original research, etc) and their research focus (eg, evaluating ChatGPT's performance in ophthalmology examinations or clinical tasks). Findings: We identified 32 articles meeting our criteria, published between January and July 2023, with a peak in June (n=12). Most were original research evaluating LLMs' proficiency in clinically related tasks (n=9). Studies demonstrated that ChatGPT-4.0 outperformed its predecessor, ChatGPT-3.5, in ophthalmology exams. Furthermore, ChatGPT excelled in constructing discharge notes (n=2), evaluating diagnoses (n=2) and answering general medical queries (n=6). However, it struggled with generating scientific articles or abstracts (n=3) and answering specific subdomain questions, especially those regarding specific treatment options (n=2). ChatGPT's performance relative to other LLMs (Google's Bard, Microsoft's Bing) varied by study design. Ethical concerns such as data hallucination (n=27), authorship (n=5) and data privacy (n=2) were frequently cited. Interpretation: While LLMs hold transformative potential for healthcare and ophthalmology, concerns over accountability, accuracy and data security remain. Future research should focus on application programming interface integration, comparative assessments of popular LLMs, their ability to interpret image-based data and the establishment of standardised evaluation frameworks.
LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content

Mohamed Bayan Kmainasi,Ali Ezzat Shahroor,Maram Hasanain,Sahinur Rahman Laskar,Naeemul Hassan,Firoj Alam

2024-10-20

Abstract:Large Language Models (LLMs) have demonstrated remarkable success as general-purpose task solvers across various fields, including NLP, healthcare, finance, and law. However, their capabilities remain limited when addressing domain-specific problems, particularly in downstream NLP tasks. Research has shown that models fine-tuned on instruction-based downstream NLP datasets outperform those that are not fine-tuned. While most efforts in this area have primarily focused on resource-rich languages like English and broad domains, little attention has been given to multilingual settings and specific domains. To address this gap, this study focuses on developing a specialized LLM, LlamaLens, for analyzing news and social media content in a multilingual context. To the best of our knowledge, this is the first attempt to tackle both domain specificity and multilinguality, with a particular focus on news and social media. Our experimental setup includes 19 tasks, represented by 52 datasets covering Arabic, English, and Hindi. We demonstrate that LlamaLens outperforms the current state-of-the-art (SOTA) on 16 testing sets, and achieves comparable performance on 10 sets. We make the models and resources publicly available for the research community.(<a class="link-external link-https" href="https://huggingface.co/QCRI" rel="external noopener nofollow">this https URL</a>)

Computation and Language,Artificial Intelligence
Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models

Seyed Amir Ahmad Safavi-Naini,Shuhaib Ali,Omer Shahab,Zahra Shahhoseini,Thomas Savage,Sara Rafiee,Jamil S Samaan,Reem Al Shabeeb,Farah Ladak,Jamie O Yang,Juan Echavarria,Sumbal Babar,Aasma Shaukat,Samuel Margolis,Nicholas P Tatonetti,Girish Nadkarni,Bara El Kurdi,Ali Soroush

2024-09-04

Abstract:Background and Aims: This study evaluates the medical reasoning performance of large language models (LLMs) and vision language models (VLMs) in gastroenterology. Methods: We used 300 gastroenterology board exam-style multiple-choice questions, 138 of which contain images to systematically assess the impact of model configurations and parameters and prompt engineering strategies utilizing GPT-3.5. Next, we assessed the performance of proprietary and open-source LLMs (versions), including GPT (3.5, 4, 4o, 4omini), Claude (3, 3.5), Gemini (1.0), Mistral, Llama (2, 3, 3.1), Mixtral, and Phi (3), across different interfaces (web and API), computing environments (cloud and local), and model precisions (with and without quantization). Finally, we assessed accuracy using a semiautomated pipeline. Results: Among the proprietary models, GPT-4o (73.7%) and Claude3.5-Sonnet (74.0%) achieved the highest accuracy, outperforming the top open-source models: Llama3.1-405b (64%), Llama3.1-70b (58.3%), and Mixtral-8x7b (54.3%). Among the quantized open-source models, the 6-bit quantized Phi3-14b (48.7%) performed best. The scores of the quantized models were comparable to those of the full-precision models Llama2-7b, Llama2--13b, and Gemma2-9b. Notably, VLM performance on image-containing questions did not improve when the images were provided and worsened when LLM-generated captions were provided. In contrast, a 10% increase in accuracy was observed when images were accompanied by human-crafted image descriptions. Conclusion: In conclusion, while LLMs exhibit robust zero-shot performance in medical reasoning, the integration of visual data remains a challenge for VLMs. Effective deployment involves carefully determining optimal model configurations, encouraging users to consider either the high performance of proprietary models or the flexible adaptability of open-source models.

Computation and Language,Artificial Intelligence
ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology

Nikita Kedia,Suvansh Sanjeev,Joshua Ong,Jay Chhablani

DOI: https://doi.org/10.1038/s41433-023-02915-z

IF: 4.4563

2024-01-04

Eye

Abstract:ChatGPT, an artificial intelligence (AI) chatbot built on large language models (LLMs), has rapidly gained popularity. The benefits and limitations of this transformative technology have been discussed across various fields, including medicine. The widespread availability of ChatGPT has enabled clinicians to study how these tools could be used for a variety of tasks such as generating differential diagnosis lists, organizing patient notes, and synthesizing literature for scientific research. LLMs have shown promising capabilities in ophthalmology by performing well on the Ophthalmic Knowledge Assessment Program, providing fairly accurate responses to questions about retinal diseases, and in generating differential diagnoses list. There are current limitations to this technology, including the propensity of LLMs to "hallucinate", or confidently generate false information; their potential role in perpetuating biases in medicine; and the challenges in incorporating LLMs into research without allowing "AI-plagiarism" or publication of false information. In this paper, we provide a balanced overview of what LLMs are and introduce some of the LLMs that have been generated in the past few years. We discuss recent literature evaluating the role of these language models in medicine with a focus on ChatGPT. The field of AI is fast-paced, and new applications based on LLMs are being generated rapidly; therefore, it is important for ophthalmologists to be aware of how this technology works and how it may impact patient care. Here, we discuss the benefits, limitations, and future advancements of LLMs in patient care and research.

ophthalmology

Inhibition of brain sodium-potassium ATPase in uremic rats.

Language Enhanced Model for Eye (LEME): An Open-Source Ophthalmology-Specific Large Language Model

Ophtha-LLaMA2: A Large Language Model for Ophthalmology

Medical education with large language models in ophthalmology: custom instructions and enhanced retrieval capabilities

Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

EyeGPT: Ophthalmic Assistant with Large Language Models

OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue

A case of IgE multiple myeloma

Utilizing Large Language Models in Ophthalmology: The Current Landscape and Challenges

Large Language Models in Ophthalmology: Potential and Pitfalls

OphGLM: An ophthalmology large language-and-vision assistant

Medical, moral and legal aspects of renal replacement therapy.

LlamaCare: A Large Medical Language Model for Enhancing Healthcare Knowledge Sharing

Development and evaluation of a large language model of ophthalmology in Chinese

Exploring large language model for next generation of artificial intelligence in ophthalmology

From Text to Tables: A Local Privacy Preserving Large Language Model for Structured Information Retrieval from Medical Documents

Me LLaMA: Foundation Large Language Models for Medical Applications

Review of emerging trends and projection of future developments in large language models research in ophthalmology

LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content

Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models

ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology