AI-driven aerial robots advance whale research

Haluk Bayram

DOI: https://doi.org/10.1126/scirobotics.adt1955

IF: 25

2024-11-01

Science Robotics

Abstract:Aerial robots assisted with artificial intelligence improve real-time wildlife monitoring of sperm whales.

robotics

What problem does this paper attempt to address?

Whale Detection Enhancement through Synthetic Satellite Images

Akshaj Gaur,Cheng Liu,Xiaomin Lin,Nare Karapetyan,Yiannis Aloimonos

DOI: https://doi.org/10.23919/OCEANS52994.2023.10337400

2024-10-01

Abstract:With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales. Modern computer vision algorithms allow us to detect whales in images in a wide range of domains, further speeding up and enhancing the monitoring process. However, these algorithms heavily rely on large training datasets, which are challenging and time-consuming to collect particularly in marine or aquatic environments. Recent advances in AI however have made it possible to synthetically create datasets for training machine learning algorithms, thus enabling new solutions that were not possible before. In this work, we present a solution - SeaDroneSim2 benchmark suite, which addresses this challenge by generating aerial, and satellite synthetic image datasets to improve the detection of whales and reduce the effort required for training data collection. We show that we can achieve a 15% performance boost on whale detection compared to using the real data alone for training, by augmenting a 10% real data. We open source both the code of the simulation platform SeaDroneSim2 and the dataset generated through it.

Computer Vision and Pattern Recognition,Robotics
Reinforcement learning–based framework for whale rendezvous via autonomous sensing robots

Ninad Jadhav,Sushmita Bhattacharya,Daniel Vogt,Yaniv Aluma,Pernille Tonessen,Akarsh Prabhakara,Swarun Kumar,Shane Gero,Robert J. Wood,Stephanie Gil

DOI: https://doi.org/10.1126/scirobotics.adn7299

IF: 25

2024-11-01

Science Robotics

Abstract:Rendezvous with sperm whales for biological observations is made challenging by their prolonged dive patterns. Here, we propose an algorithmic framework that codevelops multiagent reinforcement learning–based routing (autonomy module) and synthetic aperture radar–based very high frequency (VHF) signal–based bearing estimation (sensing module) for maximizing rendezvous opportunities of autonomous robots with sperm whales. The sensing module is compatible with low-energy VHF tags commonly used for tracking wildlife. The autonomy module leverages in situ noisy bearing measurements of whale vocalizations, VHF tags, and whale dive behaviors to enable time-critical rendezvous of a robot team with multiple whales in simulation. We conducted experiments at sea in the native habitat of sperm whales using an "engineered whale"—a speedboat equipped with a VHF-emitting tag, emulating five distinct whale tracks, with different whale motions. The sensing module shows a median bearing error of 10.55° to the tag. Using bearing measurements to the engineered whale from an acoustic sensor and our sensing module, our autonomy module gives an aggregate rendezvous success rate of 81.31% for a 500-meter rendezvous distance using three robots in postprocessing. A second class of fielded experiments that used acoustic-only bearing measurements to three untagged sperm whales showed an aggregate rendezvous success rate of 68.68% for a 1000-meter rendezvous distance using two robots in postprocessing. We further validated these algorithms with several ablation studies using a sperm whale visual encounter dataset collected by marine biologists.

robotics
Whale counting in satellite and aerial images with deep learning

Emilio Guirado,Siham Tabik,Marga L. Rivas,Domingo Alcaraz-Segura,Francisco Herrera

DOI: https://doi.org/10.1038/s41598-019-50795-9

IF: 4.6

2019-10-03

Scientific Reports

Abstract:Abstract Despite their interest and threat status, the number of whales in world’s oceans remains highly uncertain. Whales detection is normally carried out from costly sighting surveys, acoustic surveys or through high-resolution images. Since deep convolutional neural networks (CNNs) are achieving great performance in several computer vision tasks, here we propose a robust and generalizable CNN-based system for automatically detecting and counting whales in satellite and aerial images based on open data and tools. In particular, we designed a two-step whale counting approach, where the first CNN finds the input images with whale presence, and the second CNN locates and counts each whale in those images. A test of the system on Google Earth images in ten global whale-watching hotspots achieved a performance (F1-measure) of 81% in detecting and 94% in counting whales. Combining these two steps increased accuracy by 36% compared to a baseline detection model alone. Applying this cost-effective method worldwide could contribute to the assessment of whale populations to guide conservation actions. Free and global access to high-resolution imagery for conservation purposes would boost this process.

multidisciplinary sciences
Dr. Doolittle uses AI: Ethical challenges of trying to speak whale

Mark Ryan,Leonie N. Bossert

DOI: https://doi.org/10.1016/j.biocon.2024.110648

IF: 7.499

2024-06-01

Biological Conservation

Abstract:Artificial intelligence (AI) technologies are increasingly used in conservation practices, e.g., to prevent poaching or inventory wildlife. Another area of application is using AI to decode animal vocalisations to understand better—and subsequently better protect—the animals. It has already been applied to different species, including various whale species. Whales have complex vocalisations that are used for sexual selection, to coo their young, for echolocation, and as a form of communication. Scientists are deploying underwater microphones (hydrophones), robotic fish, and tags to record whale vocalisation. AI is used to identify whale vocalisation patterns, understand their meaning, and digitally recreate these sounds to communicate with them. Understanding and translating whale vocalisations into something humanly understandable aims at helping to identify their movements to protect them from ship strikes and bycatch and prevent or reduce sonar that interferes with their echolocation. Using AI holds potential benefits, but it also comes with several risks. We describe current projects that use AI to decipher the vocalisations of humpback and sperm whales (Section 3). We introduce six ethical challenges of applying AI to decode whale vocalisations and highlight what needs to be addressed to establish these practices responsibly (Section 4). These challenges are anthropomorphism, privacy rights, cultural and emotional harm to whales, technological solutionism, ineffectiveness for whale conservation, and gender bias. This paper critically evaluates the use of AI to analyse whale vocalisation, concluding that using AI to decode whale vocalisations holds many benefits for whale conservation; however, using AI to try to speak with whales is ethically problematic because of the potential emotional, physical, and cultural harm caused to whales.

ecology,environmental sciences,biodiversity conservation
Aerial Wildlife Image Repository for animal monitoring with drones in the age of artificial intelligence

Sathishkumar Samiappan,B Santhana Krishnan,Damion Dehart,Landon R Jones,Jared A Elmore,Kristine O Evans,Raymond B Iglay

DOI: https://doi.org/10.1093/database/baae070

2024-07-23

Abstract:Drones (unoccupied aircraft systems) have become effective tools for wildlife monitoring and conservation. Automated animal detection and classification using artificial intelligence (AI) can substantially reduce logistical and financial costs and improve drone surveys. However, the lack of annotated animal imagery for training AI is a critical bottleneck in achieving accurate performance of AI algorithms compared to other fields. To bridge this gap for drone imagery and help advance and standardize automated animal classification, we have created the Aerial Wildlife Image Repository (AWIR), which is a dynamic, interactive database with annotated images captured from drone platforms using visible and thermal cameras. The AWIR provides the first open-access repository for users to upload, annotate, and curate images of animals acquired from drones. The AWIR also provides annotated imagery and benchmark datasets that users can download to train AI algorithms to automatically detect and classify animals, and compare algorithm performance. The AWIR contains 6587 animal objects in 1325 visible and thermal drone images of predominantly large birds and mammals of 13 species in open areas of North America. As contributors increase the taxonomic and geographic diversity of available images, the AWIR will open future avenues for AI research to improve animal surveys using drones for conservation applications. Database URL: https://projectportal.gri.msstate.edu/awir/.
Scaling whale monitoring using deep learning: A human-in-the-loop solution for analyzing aerial datasets

Justine Boulent,Bertrand Charry,Malcolm McHugh Kennedy,Emily Tissier,Raina Fan,Marianne Marcoux,Cortney A. Watt,Antoine Gagné-Turcotte

DOI: https://doi.org/10.3389/fmars.2023.1099479

IF: 5.247

2023-03-11

Frontiers in Marine Science

Abstract:To ensure effective cetacean management and conservation policies, it is necessary to collect and rigorously analyze data about these populations. Remote sensing allows the acquisition of images over large observation areas, but due to the lack of reliable automatic analysis techniques, biologists usually analyze all images by hand. In this paper, we propose a human-in-the-loop approach to couple the power of deep learning-based automation with the expertise of biologists to develop a reliable artificial intelligence assisted annotation tool for cetacean monitoring. We tested this approach to analyze a dataset of 5334 aerial images acquired in 2017 by Fisheries and Oceans Canada to monitor belugas ( Delphinapterus leucas ) from the threatened Cumberland Sound population in Clearwater Fjord, Canada. First, we used a test subset of photographs to compare predictions obtained by the fine-tuned model to manual annotations made by three Observers, expert marine mammal biologists. With only 100 annotated images for training, the model obtained between 90% and 91.4% mutual agreement with the three Observers, exceeding the minimum inter-observer agreement of 88.6% obtained between the experts themselves. Second, this model was applied to the full dataset. The predictions were then verified by an Observer and compared to annotations made completely manually and independently by another Observer. The annotating Observer and the human-in-the-loop pipeline detected 4051 belugas in common, out of a total of 4572 detections for the Observer and 4298 for our pipeline. This experiment shows that the proposed human-in-the-loop approach is suitable for processing novel aerial datasets for beluga counting and can be used to scale cetacean monitoring. It also highlights that human observers, even experienced ones, have varied detection bias, underlining the need to discuss standardization of annotation protocols.

marine & freshwater biology
ConservationBots: Autonomous aerial robot for fast robust wildlife tracking in complex terrains

Fei Chen,Hoa Van Nguyen,David A. Taggart,Katrina Falkner,S. Hamid Rezatofighi,Damith C. Ranasinghe

DOI: https://doi.org/10.1002/rob.22270

IF: 6.385

2023-11-30

Journal of Field Robotics

Abstract:Radio tagging and tracking are fundamental to understanding the movements and habitats of wildlife in their natural environments. Today, the most widespread, widely applicable technology for gathering data relies on experienced scientists armed with handheld radio‐telemetry equipment to locate low‐power radio transmitters attached to wildlife from the ground. Although aerial robots can transform labor‐intensive conservation tasks, the realization of autonomous systems for tackling task complexities under real‐world conditions remains a challenge. We developed ConservationBots—small aerial robots for tracking multiple, dynamic, radio‐tagged wildlife. The aerial robot achieves robust localization performance and fast task completion times—significant for energy‐limited aerial systems while avoiding close encounters with potential, counterproductive disturbances to wildlife. Our approach overcomes the technical and practical problems posed by combining a lightweight sensor with new concepts: (i) planning to determine both trajectory and measurement actions guided by an information‐theoretic objective, which allows the robot to strategically select near‐instantaneous range‐only measurements to achieve faster localization, and time‐consuming sensor rotation actions to acquire bearing measurements and achieve robust tracking performance; (ii) a bearing detector more robust to noise; and (iii) a tracking algorithm formulation robust to missed and false detections experienced in real‐world conditions. We conducted extensive studies: simulations built upon complex signal propagation over high‐resolution elevation data on diverse geographical terrains; field testing; studies with wombats (Lasiorhinus latifrons; nocturnal, vulnerable species dwelling in underground warrens) and tracking comparisons with a highly experienced biologist to validate the effectiveness of our aerial robot and demonstrate the significant advantages over the manual method.

robotics
Automated body length and body condition measurements of whales from drone videos for rapid assessment of population health

Kevin Charles Bierlich,Sagar Karki,Clara N. Bird,Alan Fern,Leigh G. Torres

DOI: https://doi.org/10.1111/mms.13137

2024-05-12

Marine Mammal Science

Abstract:Monitoring body length and body condition of individuals helps determine overall population health and assess adaptation to environmental changes. Aerial photogrammetry from drone‐based videos is a valuable method for obtaining body length and body condition measurements of cetaceans. However, the laborious manual processing of drone‐based videos to select frames to measure animals ultimately delays assessment of population health and hinders conservation actions. Here, we apply deep learning methods to expedite the processing of drone‐based videos to improve efficiency of obtaining important morphological measurements of whales. We develop two user‐friendly models to automatically (1) detect and output frames containing whales from drone‐based videos ("DeteX") and (2) extract body length and body condition measurements from input frames ("XtraX"). We use drone‐based videos of gray whales to compare manual versus automated measurements (n = 86). Our results show automated methods reduced processing times by one‐ninth, while achieving similar accuracy as manual measurements (mean coefficient of variation <5%). We also demonstrate how these methods are adaptable to other species and identify remaining challenges to help further improve automated measurements in the future. Importantly, these tools greatly speed up obtaining key morphological data while maintaining accuracy, which is critical for effectively monitoring population health.

zoology,marine & freshwater biology
Toward understanding the communication in sperm whales

Jacob Andreas,Gašper Beguš,Michael M Bronstein,Roee Diamant,Denley Delaney,Shane Gero,Shafi Goldwasser,David F Gruber,Sarah de Haas,Peter Malkin,Nikolay Pavlov,Roger Payne,Giovanni Petri,Daniela Rus,Pratyusha Sharma,Dan Tchernov,Pernille Tønnesen,Antonio Torralba,Daniel Vogt,Robert J Wood

DOI: https://doi.org/10.1016/j.isci.2022.104393

IF: 5.8

2022-05-13

iScience

Abstract:Machine learning has been advancing dramatically over the past decade. Most strides are human-based applications due to the availability of large-scale datasets; however, opportunities are ripe to apply this technology to more deeply understand non-human communication. We detail a scientific roadmap for advancing the understanding of communication of whales that can be built further upon as a template to decipher other forms of animal and non-human communication. Sperm whales, with their highly developed neuroanatomical features, cognitive abilities, social structures, and discrete click-based encoding make for an excellent model for advanced tools that can be applied to other animals in the future. We outline the key elements required for the collection and processing of massive datasets, detecting basic communication units and language-like higher-level structures, and validating models through interactive playback experiments. The technological capabilities developed by such an undertaking hold potential for cross-applications in broader communities investigating non-human communication and behavioral research.
Protecting endangered megafauna through AI analysis of drone images in a low-connectivity setting: a case study from Namibia

Alice Hua,Kevin Martin,Yuzeng Shen,Nicole Chen,Catherine Mou,Maximilian Sterk,Berend Reinhard,Friedrich F Reinhard,Stephen Lee,Sky Alibhai,Zoe C Jewell

DOI: https://doi.org/10.7717/peerj.13779

IF: 3.061

2022-08-03

PeerJ

Abstract:Assessing the numbers and distribution of at-risk megafauna such as the black rhino (Diceros bicornis) is key to effective conservation, yet such data are difficult to obtain. Many current monitoring technologies are invasive to the target animals and expensive. Satellite monitoring is emerging as a potential tool for very large animals (e.g., elephant) but detecting smaller species requires higher resolution imaging. Drones can deliver the required resolution and speed of monitoring, but challenges remain in delivering automated monitoring systems where internet connectivity is unreliable or absent. This study describes a model built to run on a drone to identify in situ images of megafauna. Compared with previously reported studies, this automated detection framework has a lower hardware cost and can function with a reduced internet bandwidth requirement for local network communication. It proposes the use of a Jetson Xavier NX, onboard a Parrot Anafi drone, connected to the internet throughout the flight to deliver a lightweight web-based notification system upon detection of the target species. The GPS location with the detected target species images is sent using MQ Telemetry Transport (MQTT), a lightweight messaging protocol using a publisher/subscriber architecture for IoT devices. It provides reliable message delivery when internet connection is sporadic. We used a YOLOv5l6 object detection architecture trained to identify a bounding box for one of five objects of interest in a frame of video. At an intersection over union (IoU) threshold of 0.5, our model achieved an average precision (AP) of 0.81 for black rhino (our primary target) and 0.83 for giraffe (Giraffa giraffa). The model was less successful at identifying the other smaller objects which were not our primary targets: 0.34, 0.25, and 0.42 for ostrich (Struthio camelus australis), springbok (Antidorcas marsupialis) and human respectively. We used several techniques to optimize performance and overcome the inherent challenge of small objects (animals) in the data. Although our primary focus for the development of the model was rhino, we included other species classes to emulate field conditions where many animal species are encountered, and thus reduce the false positive occurrence rate for rhino detections. To constrain model overfitting, we trained the model on a dataset with varied terrain, angle and lighting conditions and used data augmentation techniques (i.e., GANs). We used image tiling and a relatively larger (i.e., higher resolution) image input size to compensate for the difficulty faced in detecting small objects when using YOLO. In this study, we demonstrated the potential of a drone-based AI pipeline model to automate the detection of free-ranging megafauna detection in a remote setting and create alerts to a wildlife manager in a relatively poorly connected field environment.
Remote sensing techniques for automated marine mammals detection: a review of methods and current challenges

Esteban N Rodofili,Vincent Lecours,Michelle LaRue

DOI: https://doi.org/10.7717/peerj.13540

IF: 3.061

2022-06-20

PeerJ

Abstract:Marine mammals are under pressure from multiple threats, such as global climate change, bycatch, and vessel collisions. In this context, more frequent and spatially extensive surveys for abundance and distribution studies are necessary to inform conservation efforts. Marine mammal surveys have been performed visually from land, ships, and aircraft. These methods can be costly, logistically challenging in remote locations, dangerous to researchers, and disturbing to the animals. The growing use of imagery from satellite and unoccupied aerial systems (UAS) can help address some of these challenges, complementing crewed surveys and allowing for more frequent and evenly distributed surveys, especially for remote locations. However, manual counts in satellite and UAS imagery remain time and labor intensive, but the automation of image analyses offers promising solutions. Here, we reviewed the literature for automated methods applied to detect marine mammals in satellite and UAS imagery. The performance of studies is quantitatively compared with metrics that evaluate false positives and false negatives from automated detection against manual counts of animals, which allows for a better assessment of the impact of miscounts in conservation contexts. In general, methods that relied solely on statistical differences in the spectral responses of animals and their surroundings performed worse than studies that used convolutional neural networks (CNN). Despite mixed results, CNN showed promise, and its use and evaluation should continue. Overall, while automation can reduce time and labor, more research is needed to improve the accuracy of automated counts. With the current state of knowledge, it is best to use semi-automated approaches that involve user revision of the output. These approaches currently enable the best tradeoff between time effort and detection accuracy. Based on our analysis, we identified thermal infrared UAS imagery as a future research avenue for marine mammal detection and also recommend the further exploration of object-based image analysis (OBIA). Our analysis also showed that past studies have focused on the automated detection of baleen whales and pinnipeds and that there is a gap in studies looking at toothed whales, polar bears, sirenians, and mustelids.
Unmanned aerial vehicles for surveying marine fauna: assessing detection probability

Amanda Hodgson,David Peel,Natalie Kelly

DOI: https://doi.org/10.1002/eap.1519

Abstract:Aerial surveys are conducted for various fauna to assess abundance, distribution, and habitat use over large spatial scales. They are traditionally conducted using light aircraft with observers recording sightings in real time. Unmanned Aerial Vehicles (UAVs) offer an alternative with many potential advantages, including eliminating human risk. To be effective, this emerging platform needs to provide detection rates of animals comparable to traditional methods. UAVs can also acquire new types of information, and this new data requires a reevaluation of traditional analyses used in aerial surveys; including estimating the probability of detecting animals. We conducted 17 replicate UAV surveys of humpback whales (Megaptera novaeangliae) while simultaneously obtaining a 'census' of the population from land-based observations, to assess UAV detection probability. The ScanEagle UAV, carrying a digital SLR camera, continuously captured images (with 75% overlap) along transects covering the visual range of land-based observers. We also used ScanEagle to conduct focal follows of whale pods (n = 12, mean duration = 40 min), to assess a new method of estimating availability. A comparison of the whale detections from the UAV to the land-based census provided an estimated UAV detection probability of 0.33 (CV = 0.25; incorporating both availability and perception biases), which was not affected by environmental covariates (Beaufort sea state, glare, and cloud cover). According to our focal follows, the mean availability was 0.63 (CV = 0.37), with pods including mother/calf pairs having a higher availability (0.86, CV = 0.20) than those without (0.59, CV = 0.38). The follows also revealed (and provided a potential correction for) a downward bias in group size estimates from the UAV surveys, which resulted from asynchronous diving within whale pods, and a relatively short observation window of 9 s. We have shown that UAVs are an effective alternative to traditional methods, providing a detection probability that is within the range of previous studies for our target species. We also describe a method of assessing availability bias that represents spatial and temporal characteristics of a survey, from the same perspective as the survey platform, is benign, and provides additional data on animal behavior.
An empirical study of automatic wildlife detection using drone thermal imaging and object detection

Miao Chang,Tan Vuong,Manas Palaparthi,Lachlan Howell,Alessio Bonti,Mohamed Abdelrazek,Duc Thanh Nguyen

2023-10-17

Abstract:Artificial intelligence has the potential to make valuable contributions to wildlife management through cost-effective methods for the collection and interpretation of wildlife data. Recent advances in remotely piloted aircraft systems (RPAS or ``drones'') and thermal imaging technology have created new approaches to collect wildlife data. These emerging technologies could provide promising alternatives to standard labourious field techniques as well as cover much larger areas. In this study, we conduct a comprehensive review and empirical study of drone-based wildlife detection. Specifically, we collect a realistic dataset of drone-derived wildlife thermal detections. Wildlife detections, including arboreal (for instance, koalas, phascolarctos cinereus) and ground dwelling species in our collected data are annotated via bounding boxes by experts. We then benchmark state-of-the-art object detection algorithms on our collected dataset. We use these experimental results to identify issues and discuss future directions in automatic animal monitoring using drones.

Computer Vision and Pattern Recognition,Multimedia,Robotics
Harnessing Artificial Intelligence for Wildlife Conservation

Paul Fergus,Carl Chalmers,Steve Longmore,Serge Wich

2024-08-30

Abstract:The rapid decline in global biodiversity demands innovative conservation strategies. This paper examines the use of artificial intelligence (AI) in wildlife conservation, focusing on the Conservation AI platform. Leveraging machine learning and computer vision, Conservation AI detects and classifies animals, humans, and poaching-related objects using visual spectrum and thermal infrared cameras. The platform processes this data with convolutional neural networks (CNNs) and Transformer architectures to monitor species, including those which are critically endangered. Real-time detection provides the immediate responses required for time-critical situations (e.g. poaching), while non-real-time analysis supports long-term wildlife monitoring and habitat health assessment. Case studies from Europe, North America, Africa, and Southeast Asia highlight the platform's success in species identification, biodiversity monitoring, and poaching prevention. The paper also discusses challenges related to data quality, model accuracy, and logistical constraints, while outlining future directions involving technological advancements, expansion into new geographical regions, and deeper collaboration with local communities and policymakers. Conservation AI represents a significant step forward in addressing the urgent challenges of wildlife conservation, offering a scalable and adaptable solution that can be implemented globally.

Computer Vision and Pattern Recognition,Artificial Intelligence
A Visual Aquaculture System Using a Cloud-Based Autonomous Drones

Naomi A. Ubina,Shyi-Chyi Cheng,Hung-Yuan Chen,Chin-Chun Chang,Hsun-Yu Lan

DOI: https://doi.org/10.3390/drones5040109

IF: 5.532

2021-10-02

Drones

Abstract:This paper presents a low-cost and cloud-based autonomous drone system to survey and monitor aquaculture sites. We incorporated artificial intelligence (AI) services using computer vision and combined various deep learning recognition models to achieve scalability and added functionality, in order to perform aquaculture surveillance tasks. The recognition model is embedded in the aquaculture cloud, to analyze images and videos captured by the autonomous drone. The recognition models detect people, cages, and ship vessels at the aquaculture site. The inclusion of AI functions for face recognition, fish counting, fish length estimation and fish feeding intensity provides intelligent decision making. For the fish feeding intensity assessment, the large amount of data in the aquaculture cloud can be an input for analysis using the AI feeding system to optimize farmer production and income. The autonomous drone and aquaculture cloud services are cost-effective and an alternative to expensive surveillance systems and multiple fixed-camera installations. The aquaculture cloud enables the drone to execute its surveillance task more efficiently with an increased navigation time. The mobile drone navigation app is capable of sending surveillance alerts and reports to users. Our multifeatured surveillance system, with the integration of deep-learning models, yielded high-accuracy results.
Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys

Javier Lenzi,Andrew F Barnas,Abdelrahman A ElSaid,Travis Desell,Robert F Rockwell,Susan N Ellis-Felege

DOI: https://doi.org/10.1038/s41598-023-28240-9

2023-01-18

Abstract:Imagery from drones is becoming common in wildlife research and management, but processing data efficiently remains a challenge. We developed a methodology for training a convolutional neural network model on large-scale mosaic imagery to detect and count caribou (Rangifer tarandus), compare model performance with an experienced observer and a group of naïve observers, and discuss the use of aerial imagery and automated methods for large mammal surveys. Combining images taken at 75 m and 120 m above ground level, a faster region-based convolutional neural network (Faster-RCNN) model was trained in using annotated imagery with the labels: "adult caribou", "calf caribou", and "ghost caribou" (animals moving between images, producing blurring individuals during the photogrammetry processing). Accuracy, precision, and recall of the model were 80%, 90%, and 88%, respectively. Detections between the model and experienced observer were highly correlated (Pearson: 0.96-0.99, P value < 0.05). The model was generally more effective in detecting adults, calves, and ghosts than naïve observers at both altitudes. We also discuss the need to improve consistency of observers' annotations if manual review will be used to train models accurately. Generalization of automated methods for large mammal detections will be necessary for large-scale studies with diverse platforms, airspace restrictions, and sensor capabilities.
Deep learning algorithm outperforms experienced human observer at detection of blue whale D‐calls: a double‐observer analysis

Brian S. Miller,Shyam Madhusudhana,Meghan G. Aulich,Nat Kelly

DOI: https://doi.org/10.1002/rse2.297

IF: 5.7874

2022-08-30

Remote Sensing in Ecology and Conservation

Abstract:We describe an important advance in methods for characterising automated detectors of bioacoustic data from passive acoustic monitoring stations. In addition to describing a state‐of‐the‐art automated detector based on deep‐learning, our advance comes from applying double‐observer statistical methods, commonly used in visual surveys, to compare our deep‐learning detector directly to a human analyst. To the best of our knowledge, this is the first time that an automated bioacoustics detector has been directly compared to a human observer. We also quantify, via our case study on critically endangered Antarctic blue whales, the manner in which our modern AI detector of whale sounds was superior. An automated algorithm for passive acoustic detection of blue whale D‐calls was developed based on established deep learning methods for image recognition via the DenseNet architecture. The detector was trained on annotated acoustic recordings from the Antarctic, and performance of the detector was assessed by calculating precision and recall using a separate independent dataset also from the Antarctic. Detections from both the human analyst and automated detector were then inspected by an independent judge to identify any calls missed by either approach and to adjudicate whether the apparent false‐positive detections from the automated approach were actually true positives. A final performance assessment was conducted using double‐observer methods (via a closed‐population Huggins mark–recapture model) to assess the probability of detection of calls by both the human analyst and automated detector, based on the assumption of false‐positive‐free adjudicated detections. According to our double‐observer analysis, the automated detector showed superior performance with higher recall and fewer false positives than the original human analyst, and with performance similar to existing top automated detectors. To understand the performance of both detectors we inspected the time‐series and signal‐to‐noise ratio (SNR) of detections for the test dataset, and found that most of the advantages from the automated detector occurred at low and medium SNR.

ecology,remote sensing
Seabird surveillance: combining CCTV and artificial intelligence for monitoring and research

Jonas Hentati‐Sundberg,Agnes B. Olin,Sheetal Reddy,Per‐Arvid Berglund,Erik Svensson,Mareddy Reddy,Siddharta Kasarareni,Astrid A. Carlsen,Matilda Hanes,Shreyash Kad,Olof Olsson

DOI: https://doi.org/10.1002/rse2.329

IF: 5.7874

2023-03-09

Remote Sensing in Ecology and Conservation

Abstract:Surveillance cameras combined with object detection allow for automated monitoring of breeding common murres. Phenology, breeding numbers, chick growth and disturbance event are quantified at high resolution without interfering with the birds. The system can be modified to fit various types of ecological research and monitoring goals and thereby provide up‐to‐date support for conservation and ecosystem management. Ecological research and monitoring need to be able to rapidly convey information that can form the basis of scientifically sound management. Automated sensor systems, especially if combined with artificial intelligence, can contribute to such rapid high‐resolution data retrieval. Here, we explore the prospects of automated methods to generate insights for seabirds, which are often monitored for their high conservation value and for being sentinels for marine ecosystem changes. We have developed a system of video surveillance combined with automated image processing, which we apply to common murres Uria aalge. The system uses a deep learning algorithm for object detection (YOLOv5) that has been trained on annotated images of adult birds, chicks and eggs, and outputs time, location, size and confidence level of all detections, frame‐by‐frame, in the supplied video material. A total of 144 million bird detections were generated from a breeding cliff over three complete breeding seasons (2019–2021). We demonstrate how object detection can be used to accurately monitor breeding phenology and chick growth. Our automated monitoring approach can also identify and quantify rare events that are easily missed in traditional monitoring, such as disturbances from predators. Further, combining automated video analysis with continuous measurements from a temperature logger allows us to study impacts of heat waves on nest attendance in high detail. Our automated system thus produces comparable, and in several cases significantly more detailed, data than those generated from observational field studies. By running in real time on the camera streams, it has the potential to supply researchers and managers with high‐resolution up‐to‐date information on seabird population status. We describe how the system can be modified to fit various types of ecological research and monitoring goals and thereby provide up‐to‐date support for conservation and ecosystem management.

ecology,remote sensing
Elephants and Algorithms: A Review of the Current and Future Role of AI in Elephant Monitoring

Leandra Brickson,Fritz Vollrath,Alexander J. Titus

DOI: https://doi.org/10.1098/rsif.2023.0367

2023-12-16

Abstract:Artificial intelligence (AI) and machine learning (ML) present revolutionary opportunities to enhance our understanding of animal behavior and conservation strategies. Using elephants, a crucial species in Africa's protected areas, as our focal point, we delve into the role of AI and ML in their conservation. Given the increasing amounts of data gathered from a variety of sensors like cameras, microphones, geophones, drones, and satellites, the challenge lies in managing and interpreting this vast data. New AI and ML techniques offer solutions to streamline this process, helping us extract vital information that might otherwise be overlooked. This paper focuses on the different AI-driven monitoring methods and their potential for improving elephant conservation. Collaborative efforts between AI experts and ecological researchers are essential in leveraging these innovative technologies for enhanced wildlife conservation, setting a precedent for numerous other species.

Artificial Intelligence,Machine Learning
Whale Casting: Remote mobile streaming humpback whale vocalizations to the world

James P. Crutchfield,Alexandra M. Jurgens

DOI: https://doi.org/10.48550/arXiv.2212.02462

IF: 6.4588

2022-12-05

Human-Computer Interaction

Abstract:Over several days in early August 2021, while at sea in Chatham Strait, Southeast Alaska, aboard M/Y Blue Pearl, an online twitch.tv stream broadcast in real-time humpback whale vocalizations monitored via hydrophone. Dozens on mainland North American and around the planet listened in and chatted via the stream. The webcasts demonstrated a proof-of-concept: only relatively inexpensive commercial-off-the-shelf equipment is required for remote mobile streaming at sea. These notes document what was required and make recommendations for higher-quality and larger-scale deployments. One conclusion is that real-time, automated audio documenting whale acoustic behavior is readily accessible and, using the cloud, it can be directly integrated into behavioral databases -- information sources that now often focus exclusively on nonreal-time visual-sighting narrative reports and photography.

AI-driven aerial robots advance whale research

Whale Detection Enhancement through Synthetic Satellite Images

Reinforcement learning–based framework for whale rendezvous via autonomous sensing robots

Whale counting in satellite and aerial images with deep learning

Dr. Doolittle uses AI: Ethical challenges of trying to speak whale

Aerial Wildlife Image Repository for animal monitoring with drones in the age of artificial intelligence

Scaling whale monitoring using deep learning: A human-in-the-loop solution for analyzing aerial datasets

ConservationBots: Autonomous aerial robot for fast robust wildlife tracking in complex terrains

Automated body length and body condition measurements of whales from drone videos for rapid assessment of population health

Toward understanding the communication in sperm whales

Protecting endangered megafauna through AI analysis of drone images in a low-connectivity setting: a case study from Namibia

Remote sensing techniques for automated marine mammals detection: a review of methods and current challenges

Unmanned aerial vehicles for surveying marine fauna: assessing detection probability

An empirical study of automatic wildlife detection using drone thermal imaging and object detection

Harnessing Artificial Intelligence for Wildlife Conservation

A Visual Aquaculture System Using a Cloud-Based Autonomous Drones

Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys

Deep learning algorithm outperforms experienced human observer at detection of blue whale D‐calls: a double‐observer analysis

Seabird surveillance: combining CCTV and artificial intelligence for monitoring and research

Elephants and Algorithms: A Review of the Current and Future Role of AI in Elephant Monitoring

Whale Casting: Remote mobile streaming humpback whale vocalizations to the world