Abstract:Evolution has shaped animal brains to detect sensory regularities in environmental stimuli. In addition, many species map one-dimensional quantities across sensory modalities, such as conspecific faces to voices, or high-pitched sounds to bright light. If basic patterns like repetitions and identities are frequently perceived in different sensory modalities, it could be advantageous to detect cross-modal isomorphisms, i.e. develop modality-independent representations of structural features, exploitable in visual, tactile, and auditory processing. While cross-modal mappings are common in the animal kingdom, the ability to map similar (isomorphic) structures across domains has been demonstrated in humans but no other animals. We tested cross-modal isomorphisms in two chimpanzees (Pan troglodytes). Individuals were previously trained to choose structurally 'symmetric' image sequences (two identical geometrical shapes separated by a different shape) presented beside 'edge' sequences (two identical shapes preceded or followed by a different one). Here, with no additional training, the choice between symmetric and edge visual sequences was preceded by playback of three concatenated sounds, which could be symmetric (mimicking the symmetric structure of reinforced images) or edge. The chimpanzees spontaneously detected a visual-auditory isomorphism. Response latencies in choosing symmetric sequences were shorter when presented with (structurally isomorphic) symmetric, rather than edge, sound triplets: The auditory stimuli interfered, based on their structural properties, with processing of the learnt visual rule. Crucially, the animals had neither been exposed to the acoustic sequences before the experiment, nor were they trained to associate sounds to images. Our result provides the first evidence of structure processing across modalities in a non-human species. It suggests that basic cross-modal abstraction capacities transcend linguistic abilities and might involve evolutionary ancient neural mechanisms.

Spontaneous establishing of cross-modal stimulus equivalence in a beluga whale

Understanding across the senses: cross-modal studies of cognition in cetaceans

Invariant crossmodal equivalence evokes visual imagery from sounds in rhesus monkeys

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Cross-Modal Interaction and Integration Through Stimulus-Specific Adaptation in the Thalamic Reticular Nucleus of Rats.

Evidence of social learning across symbolic cultural barriers in sperm whales

Dolphin self-talk: unusual acoustic behaviour of a solitary bottlenose dolphin

Symmetry Oriented Covert Acoustic Communication by Mimicking Humpback Whale Song

Can vocal conditioning trigger a semiotic ratchet in marmosets?

Bottlenose dolphins (Tursiops truncatus) display gaze alternation and referential communication in an impossible task

Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales

Six-month-old infants use cross-modal synchrony to identify novel communicative signals

Multisensory Integration Facilitates Perceptual Restoration of an Interrupted Call in a Species of Frog

Chimpanzees process structural isomorphisms across sensory modalities

Effects of experience and familiarity on visual attention and behaviours of bottlenose dolphins (Tursiops truncatus) in presence of humans

Cross-modal codification of images with auditory stimuli: a language for the visually impaired

Contextual and combinatorial structure in sperm whale vocalisations

Whistle variability and social acoustic interactions in bottlenose dolphins

The Long-Range Biomimetic Covert Communication Method Mimicking Large Whale

Novel Concept of Bionic Morse Coding for Mimicry Covert Underwater Communication

Mother–pup recognition mechanisms in Australia sea lion (Neophoca cinerea) using uni- and multi-modal approaches