Abstract:Researches on sequential vocalization often require analysis of vocalizations in long continuous sounds. In such studies as developmental ones or studies across generations in which days or months of vocalizations must be analyzed, methods for automatic recognition would be strongly desired. Although methods for automatic speech recognition for application purposes have been intensively studied, blindly applying them for biological purposes may not be an optimal solution. This is because, unlike human speech recognition, analysis of sequential vocalizations often requires accurate extraction of timing information. In the present study we propose automated systems suitable for recognizing birdsong, one of the most intensively investigated sequential vocalizations, focusing on the three properties of the birdsong. First, a song is a sequence of vocal elements, called notes, which can be grouped into categories. Second, temporal structure of birdsong is precisely controlled, meaning that temporal information is important in song analysis. Finally, notes are produced according to certain probabilistic rules, which may facilitate the accurate song recognition. We divided the procedure of song recognition into three sub-steps: local classification, boundary detection, and global sequencing, each of which corresponds to each of the three properties of birdsong. We compared the performances of several different ways to arrange these three steps. As results, we demonstrated a hybrid model of a deep neural network and a hidden Markov model is effective in recognizing birdsong with variable note sequences. We propose suitable arrangements of methods according to whether accurate boundary detection is needed. Also we designed the new measure to jointly evaluate the accuracy of note classification and boundary detection. Our methods should be applicable, with small modification and tuning, to the songs in other species that hold the three properties of the sequential vocalization.

Complex sequencing rules of birdsong can be explained by simple hidden Markov processes

A compact statistical model of the song syntax in Bengalese finch

Syntactic Complexity and Musical Proficiency Modulate Neural Processing of Non-Native Music

Lesions in a songbird vocal circuit increase variability in song syntax

Partially observable Markov models inferred using statistical tests reveal context-dependent syllable transitions in Bengalese finch songs

Goal-directed and flexible modulation of syllable sequence within birdsong

Automatic recognition of element classes and boundaries in the birdsong with variable sequences

A hierarchical neuronal model for generation and online recognition of birdsongs

Individual-specific associations between behavioral events and song notes shape budgerigar vocal sequences

Biophysical network modeling of temporal and stereotyped sequence propagation of neural activity in the premotor nucleus HVC

Language-like efficiency and structure in house finch song

Chance, long tails, and inference: a non-Gaussian, Bayesian theory of vocal learning in songbirds

Bird song comparison using deep learning trained from avian perceptual judgments

What is the Role of Thalamostriatal Circuits in Learning Vocal Sequences?

Changes in Ultra-Structures and Electrophysiological Properties in HVC of Untutored and Deafened Bengalese Finches Relation to Normally Reared Birds: Implications for Song Learning

Bursts from the past: Intrinsic properties link a network model to zebra finch song

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Finding the Beat: From Socially Coordinated Vocalizations in Songbirds to Rhythmic Entrainment in Humans

Canary Mating Season Songs Move Between Order and Disorder

Temporal sequences of spikes during practice code for time in a complex motor sequence