Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?

David Ilić,Gilles E. Gignac

DOI: https://doi.org/10.1016/j.intell.2024.101858

2024-09-11

Abstract:Large language models (LLMs) are advanced artificial intelligence (AI) systems that can perform a variety of tasks commonly found in human intelligence tests, such as defining words, performing calculations, and engaging in verbal reasoning. There are also substantial individual differences in LLM capacities. Given the consistent observation of a positive manifold and general intelligence factor in human samples, along with group-level factors (e.g., crystallized intelligence), we hypothesized that LLM test scores may also exhibit positive intercorrelations, which could potentially give rise to an artificial general ability (AGA) factor and one or more group-level factors. Based on a sample of 591 LLMs and scores from 12 tests aligned with fluid reasoning (Gf), domain-specific knowledge (Gkn), reading/writing (Grw), and quantitative knowledge (Gq), we found strong empirical evidence for a positive manifold and a general factor of ability. Additionally, we identified a combined Gkn/Grw group-level factor. Finally, the number of LLM parameters correlated positively with both general factor of ability and Gkn/Grw factor scores, although the effects showed diminishing returns. We interpreted our results to suggest that LLMs, like human cognitive abilities, may share a common underlying efficiency in processing information and solving problems, though whether LLMs manifest primarily achievement/expertise rather than intelligence remains to be determined. Finally, while models with greater numbers of parameters exhibit greater general cognitive-like abilities, akin to the connection between greater neuronal density and human general intelligence, other characteristics must also be involved.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper attempts to explore whether large language models (LLMs) exhibit positive correlations across various tasks similar to human cognitive abilities and to investigate the existence of an artificial general ability (AGA) factor and one or more group-level factors. Specifically:

1. **Positive Correlation of LLMs Performance**:
- Researchers aim to verify whether LLMs show positive correlations in their performance across different tasks, which might indicate the presence of an artificial general ability factor.

2. **Existence of an Artificial General Ability (AGA) Factor**:
- By analyzing the performance of LLMs across various tasks, researchers seek to determine whether there exists an artificial general ability factor similar to the human general intelligence (g factor).

3. **Group-Level Factors**:
- Researchers also aim to explore whether there are group-level factors in LLMs that are similar to those in human cognitive abilities.

4. **Relationship Between Number of Parameters and Performance**:
- Researchers further investigate the relationship between the number of parameters in LLMs and their overall capabilities to determine whether the number of parameters affects the general performance of LLMs.

Through these studies, the paper aims to understand the capability structure of LLMs and their potential connections to human cognitive abilities.