Abstract:Evolution has shaped animal brains to detect sensory regularities in environmental stimuli. In addition, many species map one-dimensional quantities across sensory modalities, such as conspecific faces to voices, or high-pitched sounds to bright light. If basic patterns like repetitions and identities are frequently perceived in different sensory modalities, it could be advantageous to detect cross-modal isomorphisms, i.e. develop modality-independent representations of structural features, exploitable in visual, tactile, and auditory processing. While cross-modal mappings are common in the animal kingdom, the ability to map similar (isomorphic) structures across domains has been demonstrated in humans but no other animals. We tested cross-modal isomorphisms in two chimpanzees (Pan troglodytes). Individuals were previously trained to choose structurally 'symmetric' image sequences (two identical geometrical shapes separated by a different shape) presented beside 'edge' sequences (two identical shapes preceded or followed by a different one). Here, with no additional training, the choice between symmetric and edge visual sequences was preceded by playback of three concatenated sounds, which could be symmetric (mimicking the symmetric structure of reinforced images) or edge. The chimpanzees spontaneously detected a visual-auditory isomorphism. Response latencies in choosing symmetric sequences were shorter when presented with (structurally isomorphic) symmetric, rather than edge, sound triplets: The auditory stimuli interfered, based on their structural properties, with processing of the learnt visual rule. Crucially, the animals had neither been exposed to the acoustic sequences before the experiment, nor were they trained to associate sounds to images. Our result provides the first evidence of structure processing across modalities in a non-human species. It suggests that basic cross-modal abstraction capacities transcend linguistic abilities and might involve evolutionary ancient neural mechanisms.

Auditory–visual intermodal matching based on individual recognition in a chimpanzee (Pan troglodytes)

Chimpanzees process structural isomorphisms across sensory modalities

Cross-modal transfer in the chimpanzee

Body part categorical matching in chimpanzees (Pan troglodytes)

Audiovisual integration facilitates monkeys’ short-term memory

Cross-modal matching in the monkey after discrete temporal lobe lesions

Vocal-visual combinations in wild chimpanzees

Humans and marmosets share similar face recognition signatures in shape-based visual face discrimination behavior

Invariant crossmodal equivalence evokes visual imagery from sounds in rhesus monkeys

The evolution of face processing in primates

Assessing the motivational value of infant video clips on chimpanzees through discrimination learning task

Recognition of Shapes across Modalities by Infants

AlphaChimp: Tracking and Behavior Recognition of Chimpanzees

Examining the Long-term Retention of Associative Stimulus Relations in Chimpanzees (Pan troglodytes)

Cross-modal Transfer of Conditional Discrimination Training in Monkeys

Seeing Two Faces Together: Preference Formation in Humans and Rhesus Macaques.

Chimpanzee voice prints? Insights from transfer learning experiments from human voices

Humans (Homo sapiens) but not baboons (Papio papio) demonstrate crossmodal pitch‐luminance correspondence

Cross-Modal Transfer of Training in Monkeys

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Applying an eye tracking technique to gibbons: First study using scanpath measurements for visual stimuli