Abstract:We present a benchmark for detecting animal vocalizations in marine passive acoustic data including a published dataset, a concrete task and metrics to report. Three deep learning models for bioacoustics are tested on this benchmark and their performances are reported. Future efforts in the field should aim at the reduction of false‐positive rates in order to assure usability of the automated techniques. Passive acoustic monitoring (PAM) is commonly used to obtain year‐round continuous data on marine soundscapes harboring valuable information on species distributions or ecosystem dynamics. This continuously increasing amount of data requires highly efficient automated analysis techniques in order to exploit the full potential of the available data. Here, we propose a benchmark, which consists of a public dataset, a well‐defined task and evaluation procedure to develop and test automated analysis techniques. This benchmark focuses on the special case of detecting animal vocalizations in a real‐world dataset from the marine realm. We believe that such a benchmark is necessary to monitor the progress in the development of new detection algorithms in the field of marine bioacoustics. We ultimately use the proposed benchmark to test three detection approaches, namely ANIMAL‐SPOT, Koogu and a simple custom sequential convolutional neural network (CNN), and report performances. We report the performance of the three detection approaches in a blocked cross‐validation fashion with 11 site‐year blocks for a multi‐species detection scenario in a large marine passive acoustic dataset. Performance was measured with three simple metrics (i.e., true classification rate, noise misclassification rate and call misclassification rate) and one combined fitness metric, which allocates more weight to the minimization of false positives created by noise. Overall, ANIMAL‐SPOT performed the best with an average fitness metric of 0.6, followed by the custom CNN with an average fitness metric of 0.57 and finally Koogu with an average fitness metric of 0.42. The presented benchmark is an important step to advance in the automatic processing of the continuously growing amount of PAM data that are collected throughout the world's oceans. To ultimately achieve usability of developed algorithms, the focus of future work should be laid on the reduction of the false positives created by noise.

Unlocking the soundscape of coral reefs with artificial intelligence: pretrained networks and unsupervised learning win out

Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics

Unsupervised clustering reveals acoustic diversity and niche differentiation in pulsed calls from a coral reef ecosystem

Cooperative-hydrophone-based Underwater Soundscape Monitoring in a Bustling Harbor Enhanced by Coral Ecosystem

Who said that? Deciphering Complex Coral Reef Soundscapes with Spatial Audio and 360 Degree Video

Interpretable and Robust Machine Learning for Exploring and Classifying Soundscape Data

Classification of animal sounds in a hyperdiverse rainforest using Convolutional Neural Networks

Visualization and quantification of coral reef soundscapes using software

Acoustic features as a tool to visualize and explore marine soundscapes: Applications illustrated using marine mammal passive acoustic monitoring datasets

Fish sounds: towards the evaluation of marine acoustic biodiversity through data-driven audio source separation

Automated detection of dolphin whistles with convolutional networks and transfer learning

Deep learning in marine bioacoustics: a benchmark for baleen whale detection

Monitoring of Coral Reefs Using Artificial Intelligence: A Feasible and Cost-Effective Approach

Applications of machine learning to identify and characterize the sounds produced by fish

Acoustic animal identification using unsupervised learning

Underwater Sound Classification Using Learning Based Methods: A Review

Acoustic Signatures of the Cerrado: Machine Learning Reveals Unique Soundscapes Across Diverse Phytogeographies

Soundscape Characterization Using Autoencoders and Unsupervised Learning

Combining three-dimensional acoustic coring and a convolutional neural network to quantify species contributions to benthic ecosystems

Cross-domain Sound Recognition for Efficient Underwater Data Analysis

Soundbay: Deep Learning Framework for Marine Mammals and Bioacoustic Research