Audio long read: How does ChatGPT 'think'? Psychology and neuroscience crack open AI large language models

Matthew Hutson,Benjamin Thompson

DOI: https://doi.org/10.1038/d41586-024-01562-y

IF: 64.8

2024-05-25

Nature

Abstract:To understand the 'brains' of LLMs, researchers are attempting to reverse-engineering artificial intelligence systems. Listen to an audio version of a recent Nature Feature.

multidisciplinary sciences

What problem does this paper attempt to address?

How ChatGPT works: a mini review

Giovanni Briganti

DOI: https://doi.org/10.1007/s00405-023-08337-7

Abstract:Objective: This paper offers a mini-review of OpenAI's language model, ChatGPT, detailing its mechanisms, applications in healthcare, and comparisons with other large language models (LLMs). Methods: The underlying technology of ChatGPT is outlined, focusing on its neural network architecture, training process, and the role of key elements such as input embedding, encoder, decoder, attention mechanism, and output projection. The advancements in GPT-4, including its capacity for internet connection and the integration of plugins for enhanced functionality are discussed. Results: ChatGPT can generate creative, coherent, and contextually relevant sentences, making it a valuable tool in healthcare for patient engagement, medical education, and clinical decision support. Yet, like other LLMs, it has limitations, including a lack of common sense knowledge, a propensity for hallucination of facts, a restricted context window, and potential privacy concerns. Conclusion: Despite the limitations, LLMs like ChatGPT offer transformative possibilities for healthcare. With ongoing research in model interpretability, common-sense reasoning, and handling of longer context windows, their potential is vast. It is crucial for healthcare professionals to remain informed about these technologies and consider their ethical integration into practice.
AI & robotics briefing: How psychology and neuroscience crack open the AI 'black box'

Katrina Krämer

DOI: https://doi.org/10.1038/d41586-024-01538-y

IF: 64.8

2024-05-23

Nature

Abstract:Researchers are borrowing approaches from psychology and neuroscience to find out how large language models 'think'. Plus, self-driving super lab finds record-breaking laser material and AI-designed drugs get a reality check. Researchers are borrowing approaches from psychology and neuroscience to find out how large language models 'think'. Plus, self-driving super lab finds record-breaking laser material and AI-designed drugs get a reality check.

multidisciplinary sciences
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Rongjie Huang,Mingze Li,Dongchao Yang,Jiatong Shi,Xuankai Chang,Zhenhui Ye,Yuning Wu,Zhiqing Hong,Jiawei Huang,Jinglin Liu,Yi Ren,Zhou Zhao,Shinji Watanabe

2023-04-26

Abstract:Large language models (LLMs) have exhibited remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. Despite the recent success, current LLMs are not capable of processing complex audio information or conducting spoken conversations (like Siri or Alexa). In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i.e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input/output interface (ASR, TTS) to support spoken dialogue. With an increasing demand to evaluate multi-modal LLMs of human intention understanding and cooperation with foundation models, we outline the principles and processes and test AudioGPT in terms of consistency, capability, and robustness. Experimental results demonstrate the capabilities of AudioGPT in solving AI tasks with speech, music, sound, and talking head understanding and generation in multi-round dialogues, which empower humans to create rich and diverse audio content with unprecedented ease. Our system is publicly available at \url{<a class="link-external link-https" href="https://github.com/AIGC-Audio/AudioGPT" rel="external noopener nofollow">this https URL</a>}.

Computation and Language,Artificial Intelligence,Sound,Audio and Speech Processing
Language Writ Large: LLMs, ChatGPT, Grounding, Meaning and Understanding

Stevan Harnad

2024-02-04

Abstract:Apart from what (little) OpenAI may be concealing from us, we all know (roughly) how ChatGPT works (its huge text database, its statistics, its vector representations, and their huge number of parameters, its next-word training, and so on). But none of us can say (hand on heart) that we are not surprised by what ChatGPT has proved to be able to do with these resources. This has even driven some of us to conclude that ChatGPT actually understands. It is not true that it understands. But it is also not true that we understand how it can do what it can do. I will suggest some hunches about benign biases: convergent constraints that emerge at LLM scale that may be helping ChatGPT do so much better than we would have expected. These biases are inherent in the nature of language itself, at LLM scale, and they are closely linked to what it is that ChatGPT lacks, which is direct sensorimotor grounding to connect its words to their referents and its propositions to their meanings. These convergent biases are related to (1) the parasitism of indirect verbal grounding on direct sensorimotor grounding, (2) the circularity of verbal definition, (3) the mirroring of language production and comprehension, (4) iconicity in propositions at LLM scale, (5) computational counterparts of human categorical perception in category learning by neural nets, and perhaps also (6) a conjecture by Chomsky about the laws of thought. The exposition will be in the form of a dialogue with ChatGPT-4.

Computation and Language,Neurons and Cognition
Large Language Models and the Reverse Turing Test

Terrence J. Sejnowski

DOI: https://doi.org/10.1162/neco_a_01563

IF: 3.278

2023-02-17

Neural Computation

Abstract:Abstract Large language models (LLMs) have been transformative. They are pretrained foundational models that are self-supervised and can be adapted with fine-tuning to a wide range of natural language tasks, each of which previously would have required a separate network model. This is one step closer to the extraordinary versatility of human language. GPT-3 and, more recently, LaMDA, both of them LLMs, can carry on dialogs with humans on many topics after minimal priming with a few examples. However, there has been a wide range of reactions and debate on whether these LLMs understand what they are saying or exhibit signs of intelligence. This high variance is exhibited in three interviews with LLMs reaching wildly different conclusions. A new possibility was uncovered that could explain this divergence. What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a reverse Turing test. If so, then by studying interviews, we may be learning more about the intelligence and beliefs of the interviewer than the intelligence of the LLMs. As LLMs become more capable, they may transform the way we interact with machines and how they interact with each other. Increasingly, LLMs are being coupled with sensorimotor devices. LLMs can talk the talk, but can they walk the walk? A road map for achieving artificial general autonomy is outlined with seven major improvements inspired by brain systems and how LLMs could in turn be used to uncover new insights into brain function.

computer science, artificial intelligence,neurosciences
ChatGPT broke the Turing test — the race is on for new ways to assess AI

Celeste Biever

DOI: https://doi.org/10.1038/d41586-023-02361-7

IF: 64.8

2023-07-26

Nature

Abstract:Large language models mimic human chatter, but scientists disagree on their ability to reason. Large language models mimic human chatter, but scientists disagree on their ability to reason.

multidisciplinary sciences
A blind spot for large language models: Supradiegetic linguistic information

Julia Witte Zimmerman,Denis Hudon,Kathryn Cramer,Jonathan St. Onge,Mikaela Fudolig,Milo Z. Trujillo,Christopher M. Danforth,Peter Sheridan Dodds

2024-05-16

Abstract:Large Language Models (LLMs) like ChatGPT reflect profound changes in the field of Artificial Intelligence, achieving a linguistic fluency that is impressively, even shockingly, human-like. The extent of their current and potential capabilities is an active area of investigation by no means limited to scientific researchers. It is common for people to frame the training data for LLMs as "text" or even "language". We examine the details of this framing using ideas from several areas, including linguistics, embodied cognition, cognitive science, mathematics, and history. We propose that considering what it is like to be an LLM like ChatGPT, as Nagel might have put it, can help us gain insight into its capabilities in general, and in particular, that its exposure to linguistic training data can be productively reframed as exposure to the diegetic information encoded in language, and its deficits can be reframed as ignorance of extradiegetic information, including supradiegetic linguistic information. Supradiegetic linguistic information consists of those arbitrary aspects of the physical form of language that are not derivable from the one-dimensional relations of context -- frequency, adjacency, proximity, co-occurrence -- that LLMs like ChatGPT have access to. Roughly speaking, the diegetic portion of a word can be thought of as its function, its meaning, as the information in a theoretical vector in a word embedding, while the supradiegetic portion of the word can be thought of as its form, like the shapes of its letters or the sounds of its syllables. We use these concepts to investigate why LLMs like ChatGPT have trouble handling palindromes, the visual characteristics of symbols, translating Sumerian cuneiform, and continuing integer sequences.

Computation and Language,Artificial Intelligence
ChatGPT in the Age of Generative AI and Large Language Models: A Concise Survey

Salman Mohamadi,Ghulam Mujtaba,Ngan Le,Gianfranco Doretto,Donald A. Adjeroh

DOI: https://doi.org/10.48550/arxiv.2307.04251

2023-07-09

Computation and Language

Abstract:ChatGPT is a large language model (LLM) created by OpenAI that has been carefully trained on a large amount of data. It has revolutionized the field of natural language processing (NLP) and has pushed the boundaries of LLM capabilities. ChatGPT has played a pivotal role in enabling widespread public interaction with generative artificial intelligence (GAI) on a large scale. It has also sparked research interest in developing similar technologies and investigating their applications and implications. In this paper, our primary goal is to provide a concise survey on the current lines of research on ChatGPT and its evolution. We considered both the glass box and black box views of ChatGPT, encompassing the components and foundational elements of the technology, as well as its applications, impacts, and implications. The glass box approach focuses on understanding the inner workings of the technology, and the black box approach embraces it as a complex system, and thus examines its inputs, outputs, and effects. This paves the way for a comprehensive exploration of the technology and provides a road map for further research and experimentation. We also lay out essential foundational literature on LLMs and GAI in general and their connection with ChatGPT. This overview sheds light on existing and missing research lines in the emerging field of LLMs, benefiting both public users and developers. Furthermore, the paper delves into the broad spectrum of applications and significant concerns in fields such as education, research, healthcare, finance, etc.
Challenging large language models' " intelligence" with human tools: A neuropsychological investigation in Italian language on prefrontal functioning

Riccardo Loconte,Graziella Orrù,Mirco Tribastone,Pietro Pietrini,Giuseppe Sartori

DOI: https://doi.org/10.1016/j.heliyon.2024.e38911

IF: 3.776

2024-10-03

Heliyon

Abstract:The Artificial Intelligence (AI) research community has used ad-hoc benchmarks to measure the "intelligence" level of Large Language Models (LLMs). In humans, intelligence is closely linked to the functional integrity of the prefrontal lobes, which are essential for higher-order cognitive processes. Previous research has found that LLMs struggle with cognitive tasks that rely on these prefrontal functions, highlighting a significant challenge in replicating human-like intelligence. In December 2022, OpenAI released ChatGPT, a new chatbot based on the GPT-3.5 model that quickly gained popularity for its impressive ability to understand and respond to human instructions, suggesting a significant step towards intelligent behaviour in AI. Therefore, to rigorously investigate LLMs' level of "intelligence," we evaluated the GPT-3.5 and GPT-4 versions through a neuropsychological assessment using tests in the Italian language routinely employed to assess prefrontal functioning in humans. The same tests were also administered to Claude2 and Llama2 to verify whether similar language models perform similarly in prefrontal tests. When using human performance as a reference, GPT-3.5 showed inhomogeneous results on prefrontal tests, with some tests well above average, others in the lower range, and others frankly impaired. Specifically, we have identified poor planning abilities and difficulty in recognising semantic absurdities and understanding others' intentions and mental states. Claude2 exhibited a similar pattern to GPT-3.5, while Llama2 performed poorly in almost all tests. These inconsistent profiles highlight how LLMs' emergent abilities do not yet mimic human cognitive functioning. The sole exception was GPT-4, which performed within the normative range for all the tasks except planning. Furthermore, we showed how standardised neuropsychological batteries developed to assess human cognitive functions may be suitable for challenging LLMs' performance.
Large language model, AI and scientific research: why ChatGPT is only the beginning

Pietro Zangrossi,Massimo Martini,Francesco Guerrini,Pasquale DE Bonis,Giannantonio Spena,Pietro ZANGROSSI,Massimo MARTINI,Francesco GUERRINI,Pasquale DE BONIS,Giannantonio SPENA

DOI: https://doi.org/10.23736/s0390-5616.23.06171-4

2024-01-01

Journal of Neurosurgical Sciences

Abstract:ChatGPT, a conversational artificial intelligence model based on the generative pre-trained transformer GPT architecture, has garnered widespread attention due to its user-friendly nature and diverse capabilities. This technology enables users of all backgrounds to effortlessly engage in human-like conversations and receive coherent and intelligible responses. Beyond casual interactions, ChatGPT offers compelling prospects for scientific research, facilitating tasks like literature review and content summarization, ultimately expediting and enhancing the academic writing process. Still, in the field of medicine and surgery, it has already shown its endless potential in many tasks (enhancing decision-making processes, aiding in surgical planning and simulation, providing real-time assistance during surgery, improving postoperative care and rehabilitation, contributing to training, education, research, and development). However, it is crucial to acknowledge the model's limitations, encompassing knowledge constraints and the potential for erroneous responses, as well as ethical and legal considerations. This paper explores the potential benefits and pitfalls of these innovative technologies in scientific research, shedding light on their transformative impact while addressing concerns surrounding their use.

surgery,clinical neurology
Thinking Fast and Slow in Large Language Models

Thilo Hagendorff,Sarah Fabi,Michal Kosinski

DOI: https://doi.org/10.1038/s43588-023-00527-x

2023-08-02

Abstract:Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Therefore, it is of great importance to evaluate their emerging abilities. In this study, we show that LLMs like GPT-3 exhibit behavior that strikingly resembles human-like intuition - and the cognitive errors that come with it. However, LLMs with higher cognitive capabilities, in particular ChatGPT and GPT-4, learned to avoid succumbing to these errors and perform in a hyperrational manner. For our experiments, we probe LLMs with the Cognitive Reflection Test (CRT) as well as semantic illusions that were originally designed to investigate intuitive decision-making in humans. Our study demonstrates that investigating LLMs with methods from psychology has the potential to reveal otherwise unknown emergent traits.

Computation and Language,Artificial Intelligence,Machine Learning
Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions

Chen Feng Tsai,Xiaochen Zhou,Sierra S. Liu,Jing Li,Mo Yu,Hongyuan Mei

2023-04-06

Abstract:Large language models (LLMs) such as ChatGPT and GPT-4 have recently demonstrated their remarkable abilities of communicating with human users. In this technical report, we take an initiative to investigate their capacities of playing text games, in which a player has to understand the environment and respond to situations by having dialogues with the game world. Our experiments show that ChatGPT performs competitively compared to all the existing systems but still exhibits a low level of intelligence. Precisely, ChatGPT can not construct the world model by playing the game or even reading the game manual; it may fail to leverage the world knowledge that it already has; it cannot infer the goal of each step as the game progresses. Our results open up new research questions at the intersection of artificial intelligence, machine learning, and natural language processing.

Computation and Language,Artificial Intelligence,Machine Learning
How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability

Ivan DeAndres-Tame,Ruben Tolosana,Ruben Vera-Rodriguez,Aythami Morales,Julian Fierrez,Javier Ortega-Garcia

DOI: https://doi.org/10.1109/ACCESS.2024.3370437

2024-02-27

Abstract:Large Language Models (LLMs) such as GPT developed by OpenAI, have already shown astonishing results, introducing quick changes in our society. This has been intensified by the release of ChatGPT which allows anyone to interact in a simple conversational way with LLMs, without any experience in the field needed. As a result, ChatGPT has been rapidly applied to many different tasks such as code- and song-writer, education, virtual assistants, etc., showing impressive results for tasks for which it was not trained (zero-shot learning).

Computer Vision and Pattern Recognition,Artificial Intelligence,Computers and Society,Machine Learning
Towards a Psychology of Machines: Large Language Models Predict Human Memory

Markus Huff,Elanur Ulakçı

2024-10-14

Abstract:Large language models (LLMs) are excelling across various tasks despite not being based on human cognition, prompting an investigation into their potential to offer insights into human cognitive mechanisms. This study examines ChatGPT's ability to predict human performance in a language-based memory task. Following theories of text comprehension, we hypothesized that recognizing ambiguous sentences is easier with relevant preceding context. Participants, including humans and ChatGPT, were given pairs of sentences: the second always a garden-path sentence, and the first providing either fitting or unfitting context. We measured their ratings of sentence relatedness and memorability. Results showed a strong alignment between ChatGPT's assessments and human memory performance. Sentences in the fitting context were rated as being more related and memorable by ChatGPT and were better remembered by humans, highlighting LLMs' potential to predict human performance and contribute to psychological theories.

Computation and Language,Artificial Intelligence
A Glimpse in ChatGPT Capabilities and its impact for AI research

Frank Joublin,Antonello Ceravola,Joerg Deigmoeller,Michael Gienger,Mathias Franzius,Julian Eggert

DOI: https://doi.org/10.48550/arXiv.2305.06087

2023-05-10

Abstract:Large language models (LLMs) have recently become a popular topic in the field of Artificial Intelligence (AI) research, with companies such as Google, Amazon, Facebook, Amazon, Tesla, and Apple (GAFA) investing heavily in their development. These models are trained on massive amounts of data and can be used for a wide range of tasks, including language translation, text generation, and question answering. However, the computational resources required to train and run these models are substantial, and the cost of hardware and electricity can be prohibitive for research labs that do not have the funding and resources of the GAFA. In this paper, we will examine the impact of LLMs on AI research. The pace at which such models are generated as well as the range of domains covered is an indication of the trend which not only the public but also the scientific community is currently experiencing. We give some examples on how to use such models in research by focusing on GPT3.5/ChatGPT3.4 and ChatGPT4 at the current state and show that such a range of capabilities in a single system is a strong sign of approaching general intelligence. Innovations integrating such models will also expand along the maturation of such AI systems and exhibit unforeseeable applications that will have important impacts on several aspects of our societies.

Artificial Intelligence,Computation and Language,Human-Computer Interaction,Machine Learning,Robotics
Towards the ultimate brain: Exploring scientific discovery with ChatGPT AI

Gerardo Adesso

DOI: https://doi.org/10.1002/aaai.12113

IF: 0.9

2023-08-28

AI Magazine

Abstract:This paper presents a novel approach to scientific discovery using an artificial intelligence (AI) environment known as ChatGPT, developed by OpenAI. This is the first paper entirely generated with outputs from ChatGPT. We demonstrate how ChatGPT can be instructed through a gamification environment to define and benchmark hypothetical physical theories. Through this environment, ChatGPT successfully simulates the creation of a new improved model, called GPT4, which combines the concepts of GPT in AI (generative pretrained transformer) and GPT in physics (generalized probabilistic theory). We show that GPT4 can use its built‐in mathematical and statistical capabilities to simulate and analyze physical laws and phenomena. As a demonstration of its language capabilities, GPT4 also generates a limerick about itself. Overall, our results demonstrate the promising potential for human‐AI collaboration in scientific discovery, as well as the importance of designing systems that effectively integrate AI's capabilities with human intelligence.

computer science, artificial intelligence
Rights and Wrongs in Talk of Mind-Reading Technology

Stephen Rainey

DOI: https://doi.org/10.1017/s0963180124000045

2024-02-17

Cambridge Quarterly of Healthcare Ethics

Abstract:This article examines the idea of mind-reading technology by focusing on an interesting case of applying a large language model (LLM) to brain data. On the face of it, experimental results appear to show that it is possible to reconstruct mental contents directly from brain data by processing via a chatGPT-like LLM. However, the author argues that this apparent conclusion is not warranted. Through examining how LLMs work, it is shown that they are importantly different from natural language. The former operates on the basis of nonrational data transformations based on a large textual corpus. The latter has a rational dimension, being based on reasons. Using this as a basis, it is argued that brain data does not directly reveal mental content, but can be processed to ground predictions indirectly about mental content. The author concludes that this is impressive but different in principle from technology-mediated mind reading. The applications of LLM-based brain data processing are nevertheless promising for speech rehabilitation or novel communication methods.

social sciences, biomedical,health care sciences & services,health policy & services
Demystifying ChatGPT: An In-depth Survey of OpenAI's Robust Large Language Models

Pronaya Bhattacharya,Vivek Kumar Prasad,Ashwin Verma,Deepak Gupta,Assadaporn Sapsomboon,Wattana Viriyasitavat,Gaurav Dhiman

DOI: https://doi.org/10.1007/s11831-024-10115-5

IF: 9.7

2024-06-20

Archives of Computational Methods in Engineering

Abstract:Recent advancements in natural language processing (NLP) have catalyzed the development of models capable of generating coherent and contextually relevant responses. Such models are applied across a diverse array of applications, including but not limited to chatbots, expert systems, question-and-answer robots, and language translation systems. Large Language Models (LLMs), exemplified by OpenAI's Generative Pretrained Transformer (GPT), have significantly transformed the NLP landscape. They have introduced unparalleled abilities in generating text that is not only contextually appropriate but also semantically rich. This evolution underscores a pivotal shift towards more sophisticated and intuitive language understanding and generation capabilities within the field. Models based on GPT are developed through extensive training on vast datasets, enabling them to grasp patterns akin to human writing styles and deliver insightful responses to intricate questions. These models excel in condensing text, extending incomplete passages, crafting imaginative narratives, and emulating conversational exchanges. However, GPT LLMs are not without their challenges, including ethical dilemmas and the propensity for disseminating misinformation. Additionally, the deployment of these models at a practical scale necessitates a substantial investment in training and computational resources, leading to concerns regarding their sustainability. ChatGPT, a variant rooted in transformer-based architectures, leverages a self-attention mechanism for data sequences and a reinforcement learning-based human feedback (RLHF) system. This enables the model to grasp long-range dependencies, facilitating the generation of contextually appropriate outputs. Despite ChatGPT marking a significant leap forward in NLP technology, there remains a lack of comprehensive discourse on its architecture, efficacy, and inherent constraints. Therefore, this survey aims to elucidate the ChatGPT model, offering an in-depth exploration of its foundational structure and operational efficacy. We meticulously examine Chat-GPT's architecture and training methodology, alongside a critical analysis of its capabilities in language generation. Our investigation reveals ChatGPT's remarkable aptitude for producing text indistinguishable from human writing, whilst also acknowledging its limitations and susceptibilities to bias. This analysis is intended to provide a clearer understanding of ChatGPT, fostering a nuanced appreciation of its contributions and challenges within the broader NLP field. We also explore the ethical and societal implications of this technology, and discuss the future of NLP and AI. Our study provides valuable insights into the inner workings of ChatGPT, and helps to shed light on the potential of LLMs for shaping the future of technology and society. The approach used as Eco-GPT, with a three-level cascade (GPT-J, J1-G, GPT-4), achieves 73% and 60% cost savings in CaseHold and CoQA datasets, outperforming GPT-4.

computer science, interdisciplinary applications,engineering, multidisciplinary,mathematics
Explaining neural activity in human listeners with deep learning via natural language processing of narrative text

Andrea G Russo,Assunta Ciarlo,Sara Ponticorvo,Francesco Di Salle,Gioacchino Tedeschi,Fabrizio Esposito

DOI: https://doi.org/10.1038/s41598-022-21782-4

2022-10-25

Abstract:Deep learning (DL) approaches may also inform the analysis of human brain activity. Here, a state-of-art DL tool for natural language processing, the Generative Pre-trained Transformer version 2 (GPT-2), is shown to generate meaningful neural encodings in functional MRI during narrative listening. Linguistic features of word unpredictability (surprisal) and contextual importance (saliency) were derived from the GPT-2 applied to the text of a 12-min narrative. Segments of variable duration (from 15 to 90 s) defined the context for the next word, resulting in different sets of neural predictors for functional MRI signals recorded in 27 healthy listeners of the narrative. GPT-2 surprisal, estimating word prediction errors from the artificial network, significantly explained the neural data in superior and middle temporal gyri (bilaterally), in anterior and posterior cingulate cortices, and in the left prefrontal cortex. GPT-2 saliency, weighing the importance of context words, significantly explained the neural data for longer segments in left superior and middle temporal gyri. These results add novel support to the use of DL tools in the search for neural encodings in functional MRI. A DL language model like the GPT-2 may feature useful data about neural processes subserving language comprehension in humans, including next-word context-related prediction.
Unraveling the mysteries of AI chatbots

Raj Bridgelall

DOI: https://doi.org/10.1007/s10462-024-10720-7

IF: 9.588

2024-03-14

Artificial Intelligence Review

Abstract:This primer provides an overview of the rapidly evolving field of generative artificial intelligence, specifically focusing on large language models like ChatGPT (OpenAI) and Bard (Google). Large language models have demonstrated unprecedented capabilities in responding to natural language prompts. The aim of this primer is to demystify the underlying theory and architecture of large language models, providing intuitive explanations for a broader audience. Learners seeking to gain insight into the technical underpinnings of large language models must sift through rapidly growing and fragmented literature on the topic. This primer brings all the main concepts into a single digestible document. Topics covered include text tokenization, vocabulary construction, token embedding, context embedding with attention mechanisms, artificial neural networks, and objective functions in model training. The primer also explores state-of-the-art methods in training large language models to generalize on specific applications and to align with human intentions. Finally, an introduction to the concept of prompt engineering highlights the importance of effective human-machine interaction through natural language in harnessing the full potential of artificial intelligence chatbots. This comprehensive yet accessible primer will benefit students and researchers seeking foundational knowledge and a deeper understanding of the inner workings of existing and emerging artificial intelligence models. The author hopes that the primer will encourage further responsible innovation and informed discussions about these increasingly powerful tools.

computer science, artificial intelligence

Audio long read: How does ChatGPT 'think'? Psychology and neuroscience crack open AI large language models

How ChatGPT works: a mini review

AI & robotics briefing: How psychology and neuroscience crack open the AI 'black box'

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Language Writ Large: LLMs, ChatGPT, Grounding, Meaning and Understanding

Large Language Models and the Reverse Turing Test

ChatGPT broke the Turing test — the race is on for new ways to assess AI

A blind spot for large language models: Supradiegetic linguistic information

ChatGPT in the Age of Generative AI and Large Language Models: A Concise Survey

Challenging large language models' " intelligence" with human tools: A neuropsychological investigation in Italian language on prefrontal functioning

Large language model, AI and scientific research: why ChatGPT is only the beginning

Thinking Fast and Slow in Large Language Models

Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions

How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability

Towards a Psychology of Machines: Large Language Models Predict Human Memory

A Glimpse in ChatGPT Capabilities and its impact for AI research

Towards the ultimate brain: Exploring scientific discovery with ChatGPT AI

Rights and Wrongs in Talk of Mind-Reading Technology

Demystifying ChatGPT: An In-depth Survey of OpenAI's Robust Large Language Models

Explaining neural activity in human listeners with deep learning via natural language processing of narrative text

Unraveling the mysteries of AI chatbots