Abstract:Changes in the frequency content of sounds over time are arguably the most basic form of information about the behavior of sound-emitting objects. In perceptual studies, such changes have mostly been investigated separately, as aspects of either pitch or timbre. Here, we propose a unitary account of "up" and "down" subjective judgments of frequency change, based on a model combining auditory correlates of acoustic cues in a sound-specific and listener-specific manner. To do so, we introduce a generalized version of so-called Shepard tones, allowing symmetric manipulations of spectral information on a fine scale, usually associated to pitch (spectral fine structure, SFS), and on a coarse scale, usually associated timbre (spectral envelope, SE). In a series of behavioral experiments, listeners reported "up" or "down" shifts across pairs of generalized Shepard tones that differed in SFS, in SE, or in both. We observed the classic properties of Shepard tones for either SFS or SE shifts: subjective judgements followed the smallest log-frequency change direction, with cases of ambiguity and circularity. Interestingly, when both SFS and SE changes were applied concurrently (synergistically or antagonistically), we observed a trade-off between cues. Listeners were encouraged to report when they perceived "both" directions of change concurrently, but this rarely happened, suggesting a unitary percept. A computational model could accurately fit the behavioral data by combining different cues reflecting frequency changes after auditory filtering. The model revealed that cue weighting depended on the nature of the sound. When presented with harmonic sounds, listeners put more weight on SFS-related cues, whereas inharmonic sounds led to more weight on SE-related cues. Moreover, these stimulus-based factors were modulated by inter-individual differences, revealing variability across listeners in the detailed recipe for "up" and "down" judgments. We argue that frequency changes are tracked perceptually via the adaptive combination of a diverse set of cues, in a manner that is in fact similar to the derivation of other basic auditory dimensions such as spatial location.

Perceived Elevation Cued by Images Rotating in Horizontal Planes

Does Perceived Angular Declination Contribute to Perceived Optical Slant on Level Ground?

Perceived Azimuth Direction is Exaggerated: Converging Evidence from Explicit and Implicit Measures

Slant Perception in Near Space is Categorically Biased: Evidence for a Vertical Tendency

Large Perceptual Distortions of Locomotor Action Space Occur in Ground-Based Coordinates: Angular Expansion and the Large-Scale Horizontal-Vertical Illusion.

Perceptual Bias in 2D Orientation is Also Present in Obliquely-Viewed Planes

Perceived slant of binocularly viewed large-scale surfaces: a common model from explicit and implicit measures.

Depth Compression Based on Mis-Scaling of Binocular Disparity May Contribute to Angular Expansion in Perceived Optical Slant.

ANGULAR OVERESTIMATION ENHANCES CODING PRECISION: SCALE EXPANSION THEORY IN SLANT PERCEPTION

Which Way is Up in the Horizontal-Vertical Illusion?

Body-centric and World-Centric Components of the Large-Scale Horizontal-Vertical Illusion.

Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution

How much I moved: Robust biases in self-rotation perception

A unitary model of auditory frequency change perception

Localization of sound presented via a spatial audio display during visually induced vection in pitch, roll, and yaw

Ventriloquism effect with sound stimuli varying in both azimuth and elevation

On Rotation Gains Within and Beyond Perceptual Limitations for Seated VR

Changes in Orientation Behavior due to Extended High-Frequency (5 to 10 kHz) Spatial Cues

Humans can use positive and negative spectrotemporal correlations to detect rising and falling pitch

The optokinetic cervico reflex during simulated helicopter flight

Tilt aftereffect from orientation discrimination learning