Abstract:The recent leap in performance of large language models (LLMs), a subclass of artificial intelligence (AI) algorithms that includes OpenAI's ChatGPT, Google Bard, and Microsoft's Copilot (formerly Bing Chat), ushered a revolution in artificial text generation. These systems, trained on billions of documents, are sophisticated enough to fool human users into thinking they are conversing with other humans [1,2]. In academia, LLM-driven chatbots have become popular tools to help draft and revise scientific text [3,4], with some going as far as including them as coauthors [5]. Enthusiasts highlight the ability of these systems to summarize entire articles, simplify jargon-laden paragraphs, and improve the clarity and conciseness of drafts, particularly for non-native English writers [6–8]. On the other hand, others have advocated for strict boundaries and restrictions [5,9,10], citing ethical and privacy concerns as well as the tendency of these tools to "hallucinate"—or confabulate and fabricate—facts and references [11]. LLMs are fed enormous amounts of information and use statistics to predict the next word in a sentence [12]. By doing so, they generate grammatically and semantically correct text in response to prompts but are unable to estimate the uncertainty or truth of their predictions—resulting in hallucinations. This also means that the generated text can be borrowed verbatim from existing sources, which has led to a growing number of copyright lawsuits [13,14]. As writers of scientific proposals, we believe that writing proposals is a very personal exercise where the final product is best when imbued with the ideas, style, and personality of the writer. The iterative process of drafting and refining also helps develop scientific writing skills [15], which are essential for a successful long-term career in academia. We also believe, however, that scientists can benefit immensely from including AI in this process, as assistants or makeshift reviewers, in particular as the algorithms that power these systems become better and more widely available. This article aims to strike a delicate balance—an enthusiastic yet cautionary tale outlining 10 best practice tips (summarized in Fig 1) for using LLMs during your grant writing journey.

Let's Get to the Point: LLM-Supported Planning, Drafting, and Revising of Research-Paper Blog Posts

OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Models

Prompting LLMs to Compose Meta-Review Drafts from Peer-Review Narratives of Scholarly Manuscripts

ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models

A publishing infrastructure for Artificial Intelligence (AI)-assisted academic authoring

Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers

Ten simple rules to leverage large language models for getting grants

Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits

ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing

Integrating Planning into Single-Turn Long-Form Text Generation

Examining Human-AI Collaboration for Co-Writing Constructive Comments Online

Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment

LLMs for Literature Review: Are we there yet?

When Young Scholars Cooperate with LLMs in Academic Tasks: The Influence of Individual Differences and Task Complexities

Paper Copilot: A Self-Evolving and Efficient LLM System for Personalized Academic Assistance

Can large language models provide useful feedback on research papers? A large-scale empirical analysis

Bridging Research and Readers: A Multi-Modal Automated Academic Papers Interpretation System

AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews

LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing

"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models