Language Writ Large: LLMs, ChatGPT, Grounding, Meaning and Understanding

Stevan Harnad

2024-02-04

Abstract:Apart from what (little) OpenAI may be concealing from us, we all know (roughly) how ChatGPT works (its huge text database, its statistics, its vector representations, and their huge number of parameters, its next-word training, and so on). But none of us can say (hand on heart) that we are not surprised by what ChatGPT has proved to be able to do with these resources. This has even driven some of us to conclude that ChatGPT actually understands. It is not true that it understands. But it is also not true that we understand how it can do what it can do. I will suggest some hunches about benign biases: convergent constraints that emerge at LLM scale that may be helping ChatGPT do so much better than we would have expected. These biases are inherent in the nature of language itself, at LLM scale, and they are closely linked to what it is that ChatGPT lacks, which is direct sensorimotor grounding to connect its words to their referents and its propositions to their meanings. These convergent biases are related to (1) the parasitism of indirect verbal grounding on direct sensorimotor grounding, (2) the circularity of verbal definition, (3) the mirroring of language production and comprehension, (4) iconicity in propositions at LLM scale, (5) computational counterparts of human categorical perception in category learning by neural nets, and perhaps also (6) a conjecture by Chomsky about the laws of thought. The exposition will be in the form of a dialogue with ChatGPT-4.

Computation and Language,Neurons and Cognition

What problem does this paper attempt to address?

What problems does this paper attempt to solve?

This paper mainly explores the capabilities and limitations of large - language models (LLMs) such as ChatGPT in processing and generating languages, especially whether these models truly "understand" the languages they process. Specifically, the paper attempts to solve the following key problems:

1. **The nature of understanding**:
- The paper discusses whether LLMs (such as ChatGPT) truly understand the languages they process. The author clearly states that these models do not have true understanding ability because they lack a direct sensorimotor basis and cannot connect symbols (words) with their referents in the real world.

2. **The Symbol Grounding Problem (SGP)**:
- The author delves into the symbol grounding problem, that is, how to make symbols (such as words) acquire their meanings in a computational system. Symbols in traditional computational models operate only based on grammatical rules and have no intrinsic meaning. In order to truly understand a language, symbols need to acquire meaning through sensory experience and interaction with the world.

3. **Different versions of the Turing test (T2 vs T3)**:
- The author distinguishes between two versions of the Turing test: T2 (limited to language ability) and T3 (including language ability and robotic sensorimotor ability). The T2 version focuses on whether a machine can simulate human language ability through text - based conversations, while the T3 version requires that a machine can not only communicate in language but also interact with the physical world through sensors and movement, so as to better understand the language.

4. **Consciousness and subjective experience (Hard Problem of Consciousness, HP)**:
- The author also discusses the hard problem of consciousness (HP), that is, to explain why and how physical processes generate subjective experiences in the brain. Even if a machine can pass the T3 test and exhibit cognitive abilities similar to those of humans, whether it will have subjective experiences remains an unsolved mystery.

5. **The difference between content words and function words**:
- The author points out that content words (such as nouns, verbs) have specific referents, while function words (such as prepositions, conjunctions) only have grammatical functions. The symbol grounding problem mainly applies to content words because they need to establish connections with objects or concepts in the real world.

6. **The meaning of definitions and propositions**:
- The author emphasizes that individual content words have referents, while propositions (such as sentences) have complete meanings. For example, "cat" refers to an animal, but "The cat is on the mat" is a proposition with a subject and a predicate and can be evaluated for its truth value.

In summary, the problems that this paper attempts to solve include: whether large - language models truly understand languages, how symbols acquire meaning in computational systems, the requirements of different versions of the Turing test for AI, the hard problem of consciousness, and the meaning and role of different types of words in languages. Through the exploration of these problems, the author hopes to reveal the limitations of current AI systems and provide directions for future research.