Abstract:Languages have long been described according to their perceived rhythmic attributes. The associated typologies are of interest in psycholinguistics as they partly predict newborns' abilities to discriminate between languages and provide insights into how adult listeners process non-native languages. Despite the relative success of rhythm metrics in supporting the existence of linguistic rhythmic classes, quantitative studies have yet to capture the full complexity of temporal regularities associated with speech rhythm. We argue that deep learning offers a powerful pattern-recognition approach to advance the characterization of the acoustic bases of speech rhythm. To explore this hypothesis, we trained a medium-sized recurrent neural network on a language identification task over a large database of speech recordings in 21 languages. The network had access to the amplitude envelopes and a variable identifying the voiced segments, assuming that this signal would poorly convey phonetic information but preserve prosodic features. The network was able to identify the language of 10-second recordings in 40% of the cases, and the language was in the top-3 guesses in two-thirds of the cases. Visualization methods show that representations built from the network activations are consistent with speech rhythm typologies, although the resulting maps are more complex than two separated clusters between stress and syllable-timed languages. We further analyzed the model by identifying correlations between network activations and known speech rhythm metrics. The findings illustrate the potential of deep learning tools to advance our understanding of speech rhythm through the identification and exploration of linguistically relevant acoustic feature spaces.

DLR : Toward a deep learned rhythmic representation for music content analysis

One Deep Music Representation to Rule Them All? : A comparative analysis of different representation learning strategies

Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization

PiRhDy: Learning Pitch-, Rhythm-, and Dynamics-aware Embeddings for Symbolic Music.

Deep Layered Learning in MIR

Representations of Sound in Deep Learning of Audio Features from Music

Modeling of the Latent Embedding of Music using Deep Neural Network

Acoustic characterization of speech rhythm: going beyond metrics with recurrent neural networks

Recognition of musical beat and style and applications in interactive humanoid robot

Polyphonic Music Modelling with LSTM-RTRBM

Music Chord Recognition Based on Midi-Trained Deep Feature and BLSTM-CRF Hybird Decoding

Deep Representation-Decoupling Neural Networks for Monaural Music Mixture Separation

N-Gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding

Music recommender using deep embedding-based features and behavior-based reinforcement learning

Deep Conditional Representation Learning for Drum Sample Retrieval by Vocalisation

Dance with Melody: An LSTM-autoencoder Approach to Music-oriented Dance Synthesis.

A Tutorial on Deep Learning for Music Information Retrieval

A Predictive Model for Music Based on Learned Interval Representations

Setting the rhythm scene: deep learning-based drum loop generation from arbitrary language cues

Text-based LSTM networks for Automatic Music Composition

Striking a New Chord: Neural Networks in Music Information Dynamics