Abstract:When recorders are used to survey acoustically conspicuous species, identification calls of the target species in recordings is essential for estimating density and abundance. We investigate how well deep neural networks identify vocalisations consisting of phrases of varying lengths, each containing a variable number of syllables. We use recordings of Hainan gibbon Nomascus hainanus vocalisations to develop and test the methods. We propose two methods for exploiting the two-level structure of such data. The first combines convolutional neural network (CNN) models with a hidden Markov model (HMM) and the second uses a convolutional recurrent neural network (CRNN). Both models learn acoustic features of syllables via a CNN and temporal correlations of syllables into phrases either via an HMM or recurrent network. We compare their performance to commonly used CNNs LeNet and VGGNet, and support vector machine (SVM). We also propose a dynamic programming method to evaluate how well phrases are predicted. This is useful for evaluating performance when vocalisations are labelled by phrases, not syllables. Our methods perform substantially better than the commonly used methods when applied to the gibbon acoustic recordings. The CRNN has an F-score of 90% on phrase prediction, which is 18% higher than the best of the SVM or LeNet and VGGNet methods. HMM post-processing raised the F-score of these last three methods to as much as 87%. The number of phrases is overestimated by CNNs and SVM, leading to error rates between 49% and 54%. With HMM, these error rates can be reduced to 0.4% at the lowest. Similarly, the error rate of CRNN's prediction is no more than 0.5%. CRNNs are better at identifying phrases of varying lengths composed of a varying number of syllables than simpler CNN or SVM models. We find a CRNN model to be best at this task, with a CNN combined with an HMM performing almost as well. We recommend that these kinds of models are used for species whose vocalisations are structured into phrases of varying lengths.

Syllable-based Speech Recognition for a Very Low-Resource Language, Chaha

Syllable based DNN-HMM Cantonese Speech to Text System

Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Language-Agnostic Syllabification with Neural Sequence Labeling

Automated Call Detection for Acoustic Surveys with Structured Calls of Varying Length

Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision

A usage of the syllable unit based on morphological statistics in Korean large vocabulary continuous speech recognition system

ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams

Gated Recurrent Units Based Hybrid Acoustic Models for Robust Speech Recognition

Acoustics Based Intent Recognition Using Discovered Phonetic Units for Low Resource Languages

Low Resource Malay Dialect Automatic Speech Recognition Modeling Using Transfer Learning from a Standard Malay Model

Investigation of Modeling Units for Mandarin Speech Recognition Using Dfsmn-ctc-smbr

Attention based on-device streaming speech recognition with large speech corpus

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model

Deep learning based large vocabulary continuous speech recognition of an under-resourced language Bangladeshi Bangla

GMM and CNN Hybrid Method for Short Utterance Speaker Recognition

Deep neural networks for syllable based acoustic modeling in Chinese speech recognition.

A comparative study on selecting acoustic modeling units in deep neural networks based large vocabulary Chinese speech recognition

Learning Syllables Using Conv-LSTM Model for Swahili Word Representation and Part-of-speech Tagging

Exploring Retraining-Free Speech Recognition for Intra-sentential Code-Switching