Abstract:The question of whether people's experience in the world shapes conceptual representation and lexical semantics is longstanding. Word-association, feature-listing and similarity rating tasks aim to address this question but require a subjective interpretation of the latent dimensions identified. In this study, we introduce a supervised representational-alignment method that (i) determines whether two groups of individuals share the same basis of a certain category, and (ii) explains in what respects they differ. In applying this method, we show that congenital blindness induces conceptual reorganization in both a-modal and sensory-related verbal domains, and we identify the associated semantic shifts. We first apply supervised feature-pruning to a language model (GloVe) to optimize prediction accuracy of human similarity judgments from word embeddings. Pruning identifies one subset of retained GloVe features that optimizes prediction of judgments made by sighted individuals and another subset that optimizes judgments made by blind. A linear probing analysis then interprets the latent semantics of these feature-subsets by learning a mapping from the retained GloVe features to 65 interpretable semantic dimensions. We applied this approach to seven semantic domains, including verbs related to motion, sight, touch, and amodal verbs related to knowledge acquisition. We find that blind individuals more strongly associate social and cognitive meanings to verbs related to motion or those communicating non-speech vocal utterances (e.g., whimper, moan). Conversely, for amodal verbs, they demonstrate much sparser information. Finally, for some verbs, representations of blind and sighted are highly similar. The study presents a formal approach for studying interindividual differences in word meaning, and the first demonstration of how blindness impacts conceptual representation of everyday verbs.

Signatures of cross-modal alignment in children's early concepts

Learning Visually Aligned Semantic Graph for Cross-Modal Manifold Matching.

Getting aligned on representational alignment

Concept Alignment

Concept Alignment as a Prerequisite for Value Alignment

The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication

Development of Multi-level Linguistic Alignment in Child-Adult Conversations

Structural alignment in incidental word learning . To appear in

Identifying and interpreting non-aligned human conceptual representations using language modeling

Aligning Machine and Human Visual Representations across Abstraction Levels

The Ungrounded Alignment Problem

Perceptual alignment contributes to referential transparency in indirect learning

Concept Formation and Alignment in Language Models: Bridging Statistical Patterns in Latent Space to Concept Taxonomy

Possible principles for aligned structure learning agents

Understanding the Learning Dynamics of Alignment with Human Feedback

Predicting Word Learning in Children from the Performance of Computer Vision Systems

Neural alignment predicts learning outcomes in students taking an introduction to computer science course

Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories

Learning Object Semantic Similarity with Self-Supervision

Evaluating alignment between humans and neural network representations in image-based learning tasks

Aligning Agents like Large Language Models