Abstract:Training deep neural networks on well-understood dependencies in speech data can provide new insights into how they learn internal representations. This paper argues that acquisition of speech can be modeled as a dependency between random space and generated speech data in the Generative Adversarial Network architecture and proposes a methodology to uncover the network's internal representations that correspond to phonetic and phonological properties. The Generative Adversarial architecture is uniquely appropriate for modeling phonetic and phonological learning because the network is trained on unannotated raw acoustic data and learning is unsupervised without any language-specific assumptions or pre-assumed levels of abstraction. A Generative Adversarial Network was trained on an allophonic distribution in English, in which voiceless stops surface as aspirated word-initially before stressed vowels, except if preceded by a sibilant [s]. The network successfully learns the allophonic alternation: the network's generated speech signal contains the conditional distribution of aspiration duration. The paper proposes a technique for establishing the network's internal representations that identifies latent variables that correspond to, for example, presence of [s] and its spectral properties. By manipulating these variables, we actively control the presence of [s] and its frication amplitude in the generated outputs. This suggests that the network learns to use latent variables as an approximation of phonetic and phonological representations. Crucially, we observe that the dependencies learned in training extend beyond the training interval, which allows for additional exploration of learning representations. The paper also discusses how the network's architecture and innovative outputs resemble and differ from linguistic behavior in language acquisition, speech disorders, and speech errors, and how well-understood dependencies in speech data can help us interpret how neural networks learn their representations.

How autosegmental is phonology?

Phonetic Ambiguity : Approaches, Touchstones, Pitfalls and New Approaches

Addressing phonological questions with ultrasound

Serialism and Opacity in Phonological Theory

Acoustic and language-specific sources for phonemic abstraction from speech

Advancements of phonetics in the 21st century: Theoretical and empirical issues in the phonetics of sound change

Advancements of phonetics in the 21st century: A critical appraisal of time and space in Articulatory Phonology

The Interplay of Morphology and Phonology

Computational Phonology

The Phonetics of Prosody

Maintenance of subcategorical information during speech perception: Revisiting misunderstood limitations

Articulatory Phonology: An Overview

Perception of Phonological Assimilation by Neural Speech Recognition Models

Phasal Syntax = Cyclic Phonology?

Autophonic Loudness of Singers in Simulated Room Acoustic Environments

How phonemes contribute to deep speaker models?

Using Mimicry to Learn about Mental Representations

Individual differences in the use of top-down versus bottom-up cues to resolve phonetic ambiguity

Differentiable Generative Phonology

Generative Adversarial Phonology: Modeling Unsupervised Phonetic and Phonological Learning With Neural Networks

Tonal Behavior as of Areal and Typological Concerns: Centering on the Sinitic and Kam-Tai Languages in Lingnan