Abstract:Objectives: Generative large language models (LLMs) are a subset of transformers-based neural network architecture models. LLMs have successfully leveraged a combination of an increased number of parameters, improvements in computational efficiency, and large pre-training datasets to perform a wide spectrum of natural language processing (NLP) tasks. Using a few examples (few-shot) or no examples (zero-shot) for prompt-tuning has enabled LLMs to achieve state-of-the-art performance in a broad range of NLP applications. This article by the American Medical Informatics Association (AMIA) NLP Working Group characterizes the opportunities, challenges, and best practices for our community to leverage and advance the integration of LLMs in downstream NLP applications effectively. This can be accomplished through a variety of approaches, including augmented prompting, instruction prompt tuning, and reinforcement learning from human feedback (RLHF). Target audience: Our focus is on making LLMs accessible to the broader biomedical informatics community, including clinicians and researchers who may be unfamiliar with NLP. Additionally, NLP practitioners may gain insight from the described best practices. Scope: We focus on 3 broad categories of NLP tasks, namely natural language understanding, natural language inferencing, and natural language generation. We review the emerging trends in prompt tuning, instruction fine-tuning, and evaluation metrics used for LLMs while drawing attention to several issues that impact biomedical NLP applications, including falsehoods in generated text (confabulation/hallucinations), toxicity, and dataset contamination leading to overfitting. We also review potential approaches to address some of these current challenges in LLMs, such as chain of thought prompting, and the phenomena of emergent capabilities observed in LLMs that can be leveraged to address complex NLP challenge in biomedical applications.

Leveraging Open-Source Large Language Models for Native Language Identification

Native Language Identification with Large Language Models

Neural Authorship Attribution: Stylometric Analysis on Large Language Models

Scaling Native Language Identification with Transformer Adapters

Applying Large Language Models for Automated Essay Scoring for Non-Native Japanese

Open, Closed, or Small Language Models for Text Classification?

Harnessing large language models' zero-shot and few-shot learning capabilities for regulatory research

On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?

Assessing the Performance of Chinese Open Source Large Language Models in Information Extraction Tasks

Large Language Models and OpenLogos: An Educational Case Scenario

Rethinking STS and NLI in Large Language Models

Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions

Origin Tracing and Detecting of LLMs

Large Language Models Struggle in Token-Level Clinical Named Entity Recognition

Can Large Language Models Identify Authorship?

Application and technology of an open source AI large language model in the medical field

Large language models for biomedicine: foundations, opportunities, challenges, and best practices

Comparison of Open-Source and Proprietary LLMs for Machine Reading Comprehension: A Practical Analysis for Industrial Applications

Distilling large language models for matching patients to clinical trials

Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models

The Accuracy of Domain Specific and Descriptive Analysis Generated by Large Language Models