Using machine learning to decode animal communication

Christian Rutz,Michael Bronstein,Aza Raskin,Sonja C Vernes,Katherine Zacarian,Damián E Blasi,Sonja C. Vernes,Damián E. Blasi

DOI: https://doi.org/10.1126/science.adg7314

IF: 56.9

2023-07-14

Science

Abstract:New methods promise transformative insights and conservation benefits

multidisciplinary sciences

What problem does this paper attempt to address?

Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales

Jacob Andreas,Gašper Beguš,Michael M. Bronstein,Roee Diamant,Denley Delaney,Shane Gero,Shafi Goldwasser,David F. Gruber,Sarah de Haas,Peter Malkin,Roger Payne,Giovanni Petri,Daniela Rus,Pratyusha Sharma,Dan Tchernov,Pernille Tønnesen,Antonio Torralba,Daniel Vogt,Robert J. Wood

DOI: https://doi.org/10.48550/arXiv.2104.08614

2021-04-17

Sound

Abstract:The past decade has witnessed a groundbreaking rise of machine learning for human language analysis, with current methods capable of automatically accurately recovering various aspects of syntax and semantics - including sentence structure and grounded word meaning - from large data collections. Recent research showed the promise of such tools for analyzing acoustic communication in nonhuman species. We posit that machine learning will be the cornerstone of future collection, processing, and analysis of multimodal streams of data in animal communication studies, including bioacoustic, behavioral, biological, and environmental data. Cetaceans are unique non-human model species as they possess sophisticated acoustic communications, but utilize a very different encoding system that evolved in an aquatic rather than terrestrial medium. Sperm whales, in particular, with their highly-developed neuroanatomical features, cognitive abilities, social structures, and discrete click-based encoding make for an excellent starting point for advanced machine learning tools that can be applied to other animals in the future. This paper details a roadmap toward this goal based on currently existing technology and multidisciplinary scientific community effort. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales, detecting their basic communication units and language-like higher-level structures, and validating these models through interactive playback experiments. The technological capabilities developed by such an undertaking are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
Toward understanding the communication in sperm whales

Jacob Andreas,Gašper Beguš,Michael M Bronstein,Roee Diamant,Denley Delaney,Shane Gero,Shafi Goldwasser,David F Gruber,Sarah de Haas,Peter Malkin,Nikolay Pavlov,Roger Payne,Giovanni Petri,Daniela Rus,Pratyusha Sharma,Dan Tchernov,Pernille Tønnesen,Antonio Torralba,Daniel Vogt,Robert J Wood

DOI: https://doi.org/10.1016/j.isci.2022.104393

IF: 5.8

2022-05-13

iScience

Abstract:Machine learning has been advancing dramatically over the past decade. Most strides are human-based applications due to the availability of large-scale datasets; however, opportunities are ripe to apply this technology to more deeply understand non-human communication. We detail a scientific roadmap for advancing the understanding of communication of whales that can be built further upon as a template to decipher other forms of animal and non-human communication. Sperm whales, with their highly developed neuroanatomical features, cognitive abilities, social structures, and discrete click-based encoding make for an excellent model for advanced tools that can be applied to other animals in the future. We outline the key elements required for the collection and processing of massive datasets, detecting basic communication units and language-like higher-level structures, and validating models through interactive playback experiments. The technological capabilities developed by such an undertaking hold potential for cross-applications in broader communities investigating non-human communication and behavioral research.
AI decodes the calls of the wild

Neil Savage

DOI: https://doi.org/10.1038/d41586-024-04050-5

IF: 64.8

2024-12-11

Nature

Abstract:Artificial intelligence could reveal how animals of the land, sea and sky talk to others of their species.

multidisciplinary sciences
Applying machine learning to primate bioacoustics: Review and perspectives

Jules Cauzinille,Benoit Favre,Ricard Marxer,Arnaud Rey

DOI: https://doi.org/10.1002/ajp.23666

2024-08-09

Abstract:This paper provides a comprehensive review of the use of computational bioacoustics as well as signal and speech processing techniques in the analysis of primate vocal communication. We explore the potential implications of machine learning and deep learning methods, from the use of simple supervised algorithms to more recent self-supervised models, for processing and analyzing large data sets obtained within the emergence of passive acoustic monitoring approaches. In addition, we discuss the importance of automated primate vocalization analysis in tackling essential questions on animal communication and highlighting the role of comparative linguistics in bioacoustic research. We also examine the challenges associated with data collection and annotation and provide insights into potential solutions. Overall, this review paper runs through a set of common or innovative perspectives and applications of machine learning for primate vocal communication analysis and outlines opportunities for future research in this rapidly developing field.
Applications of machine learning in animal behaviour studies

John Joseph Valletta,Colin Torney,Michael Kings,Alex Thornton,Joah Madden

DOI: https://doi.org/10.1016/j.anbehav.2016.12.005

IF: 3.041

2017-02-01

Animal Behaviour

Abstract:In many areas of animal behaviour research, improvements in our ability to collect large and detailed data sets are outstripping our ability to analyse them. These diverse, complex and often high-dimensional data sets exhibit nonlinear dependencies and unknown interactions across multiple variables, and may fail to conform to the assumptions of many classical statistical methods. The field of machine learning provides methodologies that are ideally suited to the task of extracting knowledge from these data. In this review, we aim to introduce animal behaviourists unfamiliar with machine learning (ML) to the promise of these techniques for the analysis of complex behavioural data. We start by describing the rationale behind ML and review a number of animal behaviour studies where ML has been successfully deployed. The ML framework is then introduced by presenting several unsupervised and supervised learning methods. Following this overview, we illustrate key ML approaches by developing data analytical pipelines for three different case studies that exemplify the types of behavioural and ecological questions ML can address. The first uses a large number of spectral and morphological characteristics that describe the appearance of pheasant, Phasianus colchicus, eggs to assign them to putative clutches. The second takes a continuous data stream of feeder visits from PIT (passive integrated transponder)-tagged jackdaws, Corvus monedula, and extracts foraging events from it, which permits the construction of social networks. Our final example uses aerial images to train a classifier that detects the presence of wildebeest, Connochaetes taurinus, to count individuals in a population. With the advent of cheaper sensing and tracking technologies an unprecedented amount of data on animal behaviour is becoming available. We believe that ML will play a central role in translating these data into scientific knowledge and become a useful addition to the animal behaviourist's analytical toolkit.

zoology,behavioral sciences
Ensemble deep learning and anomaly detection framework for automatic audio classification: Insights into deer vocalizations

Salem Ibrahim Salem,Sakae Shirayama,Sho Shimazaki,Kazuo Oki

DOI: https://doi.org/10.1016/j.ecoinf.2024.102883

IF: 5.1

2024-11-10

Ecological Informatics

Abstract:Audio recordings have emerged as a pivotal tool in field observations, enriching environmental monitoring in both the spatial and temporal dimensions. However, the richness and complexity of these recordings pose significant challenges, primarily when extracting specific sound clips from long recordings owing to the presence of ambient noise and other irrelevant sounds. Traditional methods, such as manual extraction or a sliding window over audio segments, hinder practical bioacoustic applications. Therefore, we propose a framework that begins with a robust segmentation method for extracting sound clips that potentially contain deer vocalizations. This segmentation method relies on acoustic anomaly detection and can markedly improve computational efficiency, facilitating deployment in environments with limited resources. Subsequently, the isolated clips were classified into deer and non-deer categories using machine learning models. Our investigation assessed three state-of-the-art deep learning models, ResNet50, MobileNetV2, and EfficientNet-B2, considering various hyperparameter configurations to optimize the performance. We utilized 3842 clips from two sites, Oze National Park and Taki, for training and testing. The outcomes demonstrated that all models exhibited comparable performances, with median accuracies of 98.3 % and 92.9 % during the validation and testing stages, respectively. However, no single model outperformed the others across all the evaluation metrics. For instance, ResNet50 in different configurations led to the best accuracy, F1 score, precision, and specificity, whereas MobileNetV2 had the best recall. Therefore, we adopted a consensus-based ensemble scoring system in which an audio clip was classified as a deer call when at least two of three models concurred in their classification to enhance the reliability of our classifications. Our findings demonstrated that the Ensemble approach significantly enhanced the classification performance, achieving an accuracy of 99.2 % in the test stage. The proposed approach was successfully deployed during the deer rutting seasons in Oze and Taki in 2019 and 2021, respectively. We gained invaluable insights into deer behavior by analyzing deer calls' frequency, timing, and duration. Additionally, the spatial distribution of deer calls in Taki enabled us to detect a breach in the city's protective fencing and an association between the spatial patterns of deer calls and crop damage in the two fields. We aimed to draw a comprehensive picture of deer activity, which has significant implications for both conservation efforts and understanding animal behavior in various habitats. The insights gathered from this research contribute to the scientific understanding of deer behavior and serve as a foundation for future studies and conservation initiatives. By incorporating advanced machine learning models into environmental monitoring, we have paved the way for more data-driven approaches in wildlife research.

ecology
A Theory of Unsupervised Translation Motivated by Understanding Animal Communication

Shafi Goldwasser,David F. Gruber,Adam Tauman Kalai,Orr Paradise

2023-11-04

Abstract:Neural networks are capable of translating between languages -- in some cases even between two languages where there is little or no access to parallel translations, in what is known as Unsupervised Machine Translation (UMT). Given this progress, it is intriguing to ask whether machine learning tools can ultimately enable understanding animal communication, particularly that of highly intelligent animals. We propose a theoretical framework for analyzing UMT when no parallel translations are available and when it cannot be assumed that the source and target corpora address related subject domains or posses similar linguistic structure. We exemplify this theory with two stylized models of language, for which our framework provides bounds on necessary sample complexity; the bounds are formally proven and experimentally verified on synthetic data. These bounds show that the error rates are inversely related to the language complexity and amount of common ground. This suggests that unsupervised translation of animal communication may be feasible if the communication system is sufficiently complex.

Computation and Language,Machine Learning
ANIMAL-SPOT enables animal-independent signal detection and classification using deep learning

Christian Bergler,Simeon Q Smeele,Stephen A Tyndel,Alexander Barnhill,Sara T Ortiz,Ammie K Kalan,Rachael Xi Cheng,Signe Brinkløv,Anna N Osiecka,Jakob Tougaard,Freja Jakobsen,Magnus Wahlberg,Elmar Nöth,Andreas Maier,Barbara C Klump

DOI: https://doi.org/10.1038/s41598-022-26429-y

2022-12-19

Abstract:Bioacoustic research spans a wide range of biological questions and applications, relying on identification of target species or smaller acoustic units, such as distinct call types. However, manually identifying the signal of interest is time-intensive, error-prone, and becomes unfeasible with large data volumes. Therefore, machine-driven algorithms are increasingly applied to various bioacoustic signal identification challenges. Nevertheless, biologists still have major difficulties trying to transfer existing animal- and/or scenario-related machine learning approaches to their specific animal datasets and scientific questions. This study presents an animal-independent, open-source deep learning framework, along with a detailed user guide. Three signal identification tasks, commonly encountered in bioacoustics research, were investigated: (1) target signal vs. background noise detection, (2) species classification, and (3) call type categorization. ANIMAL-SPOT successfully segmented human-annotated target signals in data volumes representing 10 distinct animal species and 1 additional genus, resulting in a mean test accuracy of 97.9%, together with an average area under the ROC curve (AUC) of 95.9%, when predicting on unseen recordings. Moreover, an average segmentation accuracy and F1-score of 95.4% was achieved on the publicly available BirdVox-Full-Night data corpus. In addition, multi-class species and call type classification resulted in 96.6% and 92.7% accuracy on unseen test data, as well as 95.2% and 88.4% regarding previous animal-specific machine-based detection excerpts. Furthermore, an Unweighted Average Recall (UAR) of 89.3% outperformed the multi-species classification baseline system of the ComParE 2021 Primate Sub-Challenge. Besides animal independence, ANIMAL-SPOT does not rely on expert knowledge or special computing resources, thereby making deep-learning-based bioacoustic signal identification accessible to a broad audience.
Using Neural Circuit Interrogation in Rodents to Unravel Human Speech Decoding

Demetrios Neophytou,Hysell V. Oviedo

DOI: https://doi.org/10.3389/fncir.2020.00002

2020-01-30

Frontiers in Neural Circuits

Abstract:The neural circuits responsible for social communication are among the least understood in the brain. Human studies have made great progress in advancing our understanding of the global computations required for processing speech, and animal models offer the opportunity to discover evolutionarily conserved mechanisms for decoding these signals. In this review article, we describe some of the most well-established speech decoding computations from human studies and describe animal research designed to reveal potential circuit mechanisms underlying these processes. Human and animal brains must perform the challenging tasks of rapidly recognizing, categorizing, and assigning communicative importance to sounds in a noisy environment. The instructions to these functions are found in the precise connections neurons make with one another. Therefore, identifying circuit-motifs in the auditory cortices and linking them to communicative functions is pivotal. We review recent advances in human recordings that have revealed the most basic unit of speech decoded by neurons is a phoneme, and consider circuit-mapping studies in rodents that have shown potential connectivity schemes to achieve this. Finally, we discuss other potentially important processing features in humans like lateralization, sensitivity to fine temporal features, and hierarchical processing. The goal is for animal studies to investigate neurophysiological and anatomical pathways responsible for establishing behavioral phenotypes that are shared between humans and animals. This can be accomplished by establishing cell types, connectivity patterns, genetic pathways and critical periods that are relevant in the development and function of social communication.

neurosciences
Our Practice Of Using Machine Learning To Recognize Species By Voice

Siddhardha Balemarthy,Atul Sajjanhar,James Xi Zheng

DOI: https://doi.org/10.48550/arXiv.1810.09078

2018-10-22

Abstract:As the technology is advancing, audio recognition in machine learning is improved as well. Research in audio recognition has traditionally focused on speech. Living creatures (especially the small ones) are part of the whole ecosystem, monitoring as well as maintaining them are important tasks. Species such as animals and birds are tending to change their activities as well as their habitats due to the adverse effects on the environment or due to other natural or man-made calamities. For those in far deserted areas, we will not have any idea about their existence until we can continuously monitor them. Continuous monitoring will take a lot of hard work and labor. If there is no continuous monitoring, then there might be instances where endangered species may encounter dangerous situations. The best way to monitor those species are through audio recognition. Classifying sound can be a difficult task even for humans. Powerful audio signals and their processing techniques make it possible to detect audio of various species. There might be many ways wherein audio recognition can be done. We can train machines either by pre-recorded audio files or by recording them live and detecting them. The audio of species can be detected by removing all the background noise and echoes. Smallest sound is considered as a syllable. Extracting various syllables is the process we are focusing on which is known as audio recognition in terms of Machine Learning (ML).

Sound,Machine Learning,Audio and Speech Processing
Modelling reindeer rut activity using on‐animal acoustic recorders and machine learning

Alexander J. Boucher,Robert B. Weladji,Øystein Holand,Jouko Kumpula

DOI: https://doi.org/10.1002/ece3.11479

IF: 3.167

2024-06-27

Ecology and Evolution

Abstract:We tested the utility of on‐animal acoustic recorders for recording animal behaviours and documenting the rutting activity of reindeer through their acoustic repertoires. We used machine learning to process the acoustic recordings of mobile animals wearing recorders. We were able to effectively use convolutional networks to describe the rutting activity of reindeer who were wearing on‐animal acoustic recorder. For decades, researchers have employed sound to study the biology of wildlife, with the aim to better understand their ecology and behaviour. By utilizing on‐animal recorders to capture audio from freely moving animals, scientists can decipher the vocalizations and glean insights into their behaviour and ecosystem dynamics through advanced signal processing. However, the laborious task of sorting through extensive audio recordings has been a major bottleneck. To expedite this process, researchers have turned to machine learning techniques, specifically neural networks, to streamline the analysis of data. Nevertheless, much of the existing research has focused predominantly on stationary recording devices, overlooking the potential benefits of employing on‐animal recorders in conjunction with machine learning. To showcase the synergy of on‐animal recorders and machine learning, we conducted a study at the Kutuharju research station in Kaamanen, Finland, where the vocalizations of rutting reindeer were recorded during their mating season. By attaching recorders to seven male reindeer during the rutting periods of 2019 and 2020, we trained convolutional neural networks to distinguish reindeer grunts with a 95% accuracy rate. This high level of accuracy allowed us to examine the reindeers' grunting behaviour, revealing patterns indicating that older, heavier males vocalized more compared to their younger, lighter counterparts. The success of this study underscores the potential of on‐animal acoustic recorders coupled with machine learning techniques as powerful tools for wildlife research, hinting at their broader applications with further advancement and optimization.

ecology,evolutionary biology
Non-verbal effecting - animal research sheds light on human emotion communication

Annett Schirmer,Ilona Croy,Katja Liebal,Stefan R Schweinberger

DOI: https://doi.org/10.1111/brv.13140

2024-09-11

Abstract:Cracking the non-verbal "code" of human emotions has been a chief interest of generations of scientists. Yet, despite much effort, a dictionary that clearly maps non-verbal behaviours onto meaning remains elusive. We suggest this is due to an over-reliance on language-related concepts and an under-appreciation of the evolutionary context in which a given non-verbal behaviour emerged. Indeed, work in other species emphasizes non-verbal effects (e.g. affiliation) rather than meaning (e.g. happiness) and differentiates between signals, for which communication benefits both sender and receiver, and cues, for which communication does not benefit senders. Against this backdrop, we develop a "non-verbal effecting" perspective for human research. This perspective extends the typical focus on facial expressions to a broadcasting of multisensory signals and cues that emerge from both social and non-social emotions. Moreover, it emphasizes the consequences or effects that signals and cues have for individuals and their social interactions. We believe that re-directing our attention from verbal emotion labels to non-verbal effects is a necessary step to comprehend scientifically how humans share what they feel.
Animal Behavior Analysis Methods Using Deep Learning: A Survey

Edoardo Fazzari,Donato Romano,Fabrizio Falchi,Cesare Stefanini

2024-05-23

Abstract:Animal behavior serves as a reliable indicator of the adaptation of organisms to their environment and their overall well-being. Through rigorous observation of animal actions and interactions, researchers and observers can glean valuable insights into diverse facets of their lives, encompassing health, social dynamics, ecological relationships, and neuroethological dimensions. Although state-of-the-art deep learning models have demonstrated remarkable accuracy in classifying various forms of animal data, their adoption in animal behavior studies remains limited. This survey article endeavors to comprehensively explore deep learning architectures and strategies applied to the identification of animal behavior, spanning auditory, visual, and audiovisual methodologies. Furthermore, the manuscript scrutinizes extant animal behavior datasets, offering a detailed examination of the principal challenges confronting this research domain. The article culminates in a comprehensive discussion of key research directions within deep learning that hold potential for advancing the field of animal behavior studies.

Machine Learning
Same data, different results? Evaluating machine learning approaches for individual identification in animal vocalisations

K Wierucka,D Murphy,SK Watson,N Falk,C Fichtel,J León,ST Leu,PM Kappeler,EF Briefer,MB Manser,N Phaniraj,M Scheumann,JM Burkart

DOI: https://doi.org/10.1101/2024.04.14.589403

2024-04-14

Abstract:Automated acoustic analysis is increasingly used in animal communication studies, and determining caller identity is a key element for many investigations. However, variability in feature extraction and classification methods limits the comparability of results across species and studies, constraining conclusions we can draw about the ecology and evolution of the groups under study. We investigated the impact of using different feature extraction (spectro-temporal measurements, Mel-frequency cepstral coefficients, and highly comparative time-series analysis) and classification methods (discriminant function analysis, support vector machines, Gaussian mixture models, neural networks, and random forests) on the consistency of classification accuracy across 16 mammalian datasets. We found that Mel-frequency cepstral coefficients and random forests yield consistently reliable results across datasets, facilitating a standardised approach across species that generates directly comparable data. These findings remained consistent across vocalisation sample sizes and number of individuals considered. We offer guidelines for processing and analysing mammalian vocalisations, fostering greater comparability, and advancing our understanding of the evolutionary significance of acoustic communication in diverse mammalian species.

Animal Behavior and Cognition
A Comprehensive Survey of Animal Identification: Exploring Data Sources, AI Advances, Classification Obstacles and the Role of Taxonomy

Qianqian Zhang,Khandakar Ahmed,Nalin Sharda,Hua Wang

DOI: https://doi.org/10.1155/2024/7033535

IF: 8.993

2024-10-13

International Journal of Intelligent Systems

Abstract:With the rapid development of entity recognition technology, animal recognition has gradually become essential in modern society, supporting labour‐intensive agriculture and animal husbandry tasks. Severe problems such as maintaining biodiversity can also benefit from animal identification technology. However, certain invasive recognition systems have resulted in permanent harm to animals, while noninvasive identification methods also exhibit certain drawbacks. This paper conducts a systematic literature review (SLR), presenting a comprehensive overview of various animal recognition technologies and their applications. Specifically, it examines methodologies such as deep learning, image processing and acoustic analysis used for different animal characteristics and identification purposes. The contribution of machine learning to animal feature extraction is highlighted, emphasising its significance for animal taxonomy and wild species monitoring. Additionally, this review addresses the challenges and limitations of current technologies, including data scarcity, model accuracy and computational requirements, and suggests opportunities for future research to overcome these obstacles.

computer science, artificial intelligence
Discrimination between the facial gestures of vocalising and non-vocalising lemurs and small apes using deep learning

Filippo Carugati,Olivier Friard,Elisa Protopapa,Camilla Mancassola,Emanuela Rabajoli,Chiara De Gregorio,Daria Valente,Valeria Ferrario,Walter Cristiano,Teresa Raimondi,Valeria Torti,Brice Lefaux,Longondraza Miaretsoa,Cristina Giacoma,Marco Gamba

DOI: https://doi.org/10.1016/j.ecoinf.2024.102847

IF: 5.1

2024-10-11

Ecological Informatics

Abstract:Facial expression studies in animal communication are essential. However, manual inspection methods are only practical for small datasets. Deep learning techniques can help discriminate facial configurations associated with vocalisations over large datasets. We extracted and labelled frames of different primate species, trained deep-learning models to identify key points on their faces, and computed distances between them to identify facial gestures. We used machine learning algorithms to classify vocalised and non-vocalised gestures across different species. The algorithms showed higher-than-chance correct classification rates, with some exceeding 90 %. Our work employs deep learning to map primate facial gestures and offers an innovative application of pose estimation systems. Our approach facilitates the investigation of facial repertoire across primate species and behavioural contexts, enabling comparative research in primate communication.

ecology
Decoding Poultry Vocalizations - Natural Language Processing and Transformer Models for Semantic and Emotional Analysis

Venkatraman Manikandan,Suresh Neethirajan

DOI: https://doi.org/10.1101/2024.12.18.629057

2024-12-20

Abstract:Deciphering the acoustic language of chickens offers new opportunities in animal welfare and ecological informatics. Their subtle vocal signals encode health conditions, emotional states, and dynamic interactions within ecosystems. Understanding the semantics of these calls provides a valuable tool for interpreting their functional vocabulary and clarifying how each sound serves a specific purpose in social and environmental contexts. We apply advanced Natural Language Processing and transformer based models to translate bioacoustic data into meaningful insights. Our method integrates Wave2Vec 2.0 for raw audio feature extraction with a fine tuned Bidirectional Encoder Representations from Transformers model, pretrained on a broad corpus of animal sounds and adapted to poultry tasks. This pipeline decodes poultry vocalizations into interpretable categories including distress calls, feeding signals, and mating vocalizations, revealing emotional nuances often overlooked by conventional analyses. Achieving 92 percent accuracy in classifying key vocalization types, our approach demonstrates the feasibility of real time automated monitoring of flock health and stress. By tracking this functional vocabulary, farmers can respond proactively to environmental or behavioral changes, improving poultry welfare, reducing stress related productivity losses, and supporting more sustainable farm management. Beyond agriculture, this research enhances our understanding of computational ecology. Accessing the semantic foundation of animal calls may indicate biodiversity, environmental stressors, and species interactions, informing integrative ecosystem level decision making.

Animal Behavior and Cognition
Harnessing Artificial Intelligence for Wildlife Conservation

Paul Fergus,Carl Chalmers,Steve Longmore,Serge Wich

2024-08-30

Abstract:The rapid decline in global biodiversity demands innovative conservation strategies. This paper examines the use of artificial intelligence (AI) in wildlife conservation, focusing on the Conservation AI platform. Leveraging machine learning and computer vision, Conservation AI detects and classifies animals, humans, and poaching-related objects using visual spectrum and thermal infrared cameras. The platform processes this data with convolutional neural networks (CNNs) and Transformer architectures to monitor species, including those which are critically endangered. Real-time detection provides the immediate responses required for time-critical situations (e.g. poaching), while non-real-time analysis supports long-term wildlife monitoring and habitat health assessment. Case studies from Europe, North America, Africa, and Southeast Asia highlight the platform's success in species identification, biodiversity monitoring, and poaching prevention. The paper also discusses challenges related to data quality, model accuracy, and logistical constraints, while outlining future directions involving technological advancements, expansion into new geographical regions, and deeper collaboration with local communities and policymakers. Conservation AI represents a significant step forward in addressing the urgent challenges of wildlife conservation, offering a scalable and adaptable solution that can be implemented globally.

Computer Vision and Pattern Recognition,Artificial Intelligence
Decoding speech from spike-based neural population recordings in secondary auditory cortex of non-human primates

Christopher Heelan,Jihun Lee,Ronan O’Shea,Laurie Lynch,David M. Brandman,Wilson Truccolo,Arto V. Nurmikko

DOI: https://doi.org/10.1038/s42003-019-0707-9

IF: 6.548

2019-12-01

Communications Biology

Abstract:Abstract Direct electronic communication with sensory areas of the neocortex is a challenging ambition for brain-computer interfaces. Here, we report the first successful neural decoding of English words with high intelligibility from intracortical spike-based neural population activity recorded from the secondary auditory cortex of macaques. We acquired 96-channel full-broadband population recordings using intracortical microelectrode arrays in the rostral and caudal parabelt regions of the superior temporal gyrus (STG). We leveraged a new neural processing toolkit to investigate the choice of decoding algorithm, neural preprocessing, audio representation, channel count, and array location on neural decoding performance. The presented spike-based machine learning neural decoding approach may further be useful in informing future encoding strategies to deliver direct auditory percepts to the brain as specific patterns of microstimulation.

biology
Characterizing Animal Behavior through Audio and Video Signal Processing

D. Valente,Haibin Wang,P. Andrews,P.P. Mitra,S. Saar,O. Tchernichovski,I. Golani,Y. Benjamini,Dan Valente,Peter Andrews,Partha P. Mitra,Sigal Saar,Ofer Tchernichovski,Ilan Golani,Yoav Benjamini

DOI: https://doi.org/10.1109/mmul.2007.71

IF: 3.4911

2007-10-01

IEEE Multimedia

Abstract:This article presents two instances in which multimedia systems and processing have elucidated animal behavior and have been central in developing quantitative descriptions. These examples demonstrate multimedia systems' utility and necessity in developing a complete phenotypic description. We hope that this article will spur interest in this subject in the multimedia community, so more advanced processing techniques will enter the field of quantitative neuroethology. You might have noticed that in our two examples, there was nothing very multimodal about the media techniques used. Both of these systems are transparently unimodal. This speaks to the limited crossover between the multimedia community and the behavioral neuroscientists (or neuroethologists). These examples did show, however, that the neuroscientific community can benefit greatly from incorporating multimedia techniques into their experiments and data analysis. As the walls between these disciplines begin to fall, experimental setups that are truly multimedia will likely appear. Such systems will allow complete phenotypic descriptions of animals in ethologically relevant settings, along with methods for analyzing, manipulating, annotating, and storing the resulting data. Combining these phenotypic descriptions with the corresponding genetic and neural network properties will facilitate the connection of these organization levels and lead to a more thorough understanding of brain functioning.

computer science, information systems, theory & methods, software engineering, hardware & architecture

Using machine learning to decode animal communication

Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales

Toward understanding the communication in sperm whales

AI decodes the calls of the wild

Applying machine learning to primate bioacoustics: Review and perspectives

Applications of machine learning in animal behaviour studies

Ensemble deep learning and anomaly detection framework for automatic audio classification: Insights into deer vocalizations

A Theory of Unsupervised Translation Motivated by Understanding Animal Communication

ANIMAL-SPOT enables animal-independent signal detection and classification using deep learning

Using Neural Circuit Interrogation in Rodents to Unravel Human Speech Decoding

Our Practice Of Using Machine Learning To Recognize Species By Voice

Modelling reindeer rut activity using on‐animal acoustic recorders and machine learning

Non-verbal effecting - animal research sheds light on human emotion communication

Animal Behavior Analysis Methods Using Deep Learning: A Survey

Same data, different results? Evaluating machine learning approaches for individual identification in animal vocalisations

A Comprehensive Survey of Animal Identification: Exploring Data Sources, AI Advances, Classification Obstacles and the Role of Taxonomy

Discrimination between the facial gestures of vocalising and non-vocalising lemurs and small apes using deep learning

Decoding Poultry Vocalizations - Natural Language Processing and Transformer Models for Semantic and Emotional Analysis

Harnessing Artificial Intelligence for Wildlife Conservation

Decoding speech from spike-based neural population recordings in secondary auditory cortex of non-human primates

Characterizing Animal Behavior through Audio and Video Signal Processing