Abstract:The conceptualization and categorization of natural objects in the human mind have long intrigued cognitive scientists and neuroscientists, offering crucial insights into human perception and cognition. Recently, the rapid development of Large Language Models (LLMs) has raised the attractive question of whether these models can also develop human-like object representations through exposure to vast amounts of linguistic and multimodal data. In this study, we combined behavioral and neuroimaging analysis methods to uncover how the object concept representations in LLMs correlate with those of humans. By collecting large-scale datasets of 4.7 million triplet judgments from LLM and Multimodal LLM (MLLM), we were able to derive low-dimensional embeddings that capture the underlying similarity structure of 1,854 natural objects. The resulting 66-dimensional embeddings were found to be highly stable and predictive, and exhibited semantic clustering akin to human mental representations. Interestingly, the interpretability of the dimensions underlying these embeddings suggests that LLM and MLLM have developed human-like conceptual representations of natural objects. Further analysis demonstrated strong alignment between the identified model embeddings and neural activity patterns in many functionally defined brain ROIs (e.g., EBA, PPA, RSC and FFA). This provides compelling evidence that the object representations in LLMs, while not identical to those in the human, share fundamental commonalities that reflect key schemas of human conceptual knowledge. This study advances our understanding of machine intelligence and informs the development of more human-like artificial cognitive systems.

Semantic Feature Verification in FLAN-T5

Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study

Formal Semantics and Verification for Feature Modeling

Conceptual structure coheres in human cognition but not in large language models

Semantic Consistency for Assuring Reliability of Large Language Models

Cognitive Modeling of Semantic Fluency Using Transformers

From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

A Semantic Web Approach to Feature Modeling and Verification

The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities

Human-like object concept representations emerge naturally in multimodal large language models

Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models

A Semantic Feature Language for Concurrent Engineering

Semantic Importance-Aware Communications with Semantic Correction Using Large Language Models

Large Language Models Are In-Context Semantic Reasoners Rather Than Symbolic Reasoners

Concept-based Analysis of Neural Networks via Vision-Language Models

With Ears to See and Eyes to Hear: Sound Symbolism Experiments with Multimodal Large Language Models

VISLA Benchmark: Evaluating Embedding Sensitivity to Semantic and Lexical Alterations

Using Conceptual Norms for Metaphor Detection

Semantic Structure-Mapping in LLM and Human Analogical Reasoning

Measuring Moral Inconsistencies in Large Language Models

Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field