Abstract:We report associations between vowel sounds, graphemes, and colors collected online from over 1,000 Dutch speakers. We also provide open materials, including a Python implementation of the structure measure and code for a single-page web application to run simple cross-modal tasks. We also provide a full dataset of color–vowel associations from 1,164 participants, including over 200 synesthetes identified using consistency measures. Our analysis reveals salient patterns in the cross-modal associations and introduces a novel measure of isomorphism in cross-modal mappings. We found that, while the acoustic features of vowels significantly predict certain mappings (replicating prior work), both vowel phoneme category and grapheme category are even better predictors of color choice. Phoneme category is the best predictor of color choice overall, pointing to the importance of phonological representations in addition to acoustic cues. Generally, high/front vowels are lighter, more green, and more yellow than low/back vowels. Synesthetes respond more strongly on some dimensions, choosing lighter and more yellow colors for high and mid front vowels than do nonsynesthetes. We also present a novel measure of cross-modal mappings adapted from ecology, which uses a simulated distribution of mappings to measure the extent to which participants’ actual mappings are structured isomorphically across modalities. Synesthetes have mappings that tend to be more structured than nonsynesthetes’, and more consistent color choices across trials correlate with higher structure scores. Nevertheless, the large majority (~ 70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of the mappings varies across individuals. Overall, this novel structure measure suggests a distribution of structured cross-modal association in the population, with synesthetes at one extreme and participants with unstructured associations at the other.

Stimulus Parameters Underlying Sound-Symbolic Mapping of Auditory Pseudowords to Visual Shapes

Differential contributions of the spectro-temporal and vocal characteristics of auditory pseudowords to multiple sound-symbolic mappings

Neural Basis of the Sound-Symbolic Crossmodal Correspondence Between Auditory Pseudowords and Visual Shapes

Phonetic underpinnings of sound symbolism across multiple domains of meaning

Eye movements reveal persistent sensitivity to sound symbolism during word learning.

Assessing sound symbolism: Investigating phonetic forms, visual shapes and letter fonts in an implicit bouba-kiki experimental paradigm

Sound symbolism shapes the English language: The maluma/takete effect in English nouns

Cross-sensory correspondences and symbolism in spoken and written language.

Semantic-based Sound Retrieval by ERP in Rapid Serial Auditory Presentation Paradigm.

The Sound Symbolism of Names

With Ears to See and Eyes to Hear: Sound Symbolism Experiments with Multimodal Large Language Models

Seeing Sounds, Hearing Shapes: a gamified study to evaluate sound-sketches

Brain networks underlying the processing of sound symbolism related to softness perception

Sound symbolism processing is lateralized to the right temporal region in the prelinguistic infant brain

Phonetic bases of sound symbolism: A review

Guessing Meaning From Word Sounds of Unfamiliar Languages: A Cross-Cultural Sound Symbolism Study

The language of sounds unheard: Exploring musical timbre semantics of large language models

Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample

Speech Perception Difficulty Modulates Theta-Band Encoding of Articulatory Synergies

Statistical relationships between surface form and sensory meanings of English words influence lexical processing.

An Overview of the Neural Circuitry for Sound Processing