Bee Tracker—an open‐source machine learning‐based video analysis software for the assessment of nesting and foraging performance of cavity‐nesting solitary bees

Anina C. Knauer,Johannes Gallmann,Matthias Albrecht

DOI: https://doi.org/10.1002/ece3.8575

IF: 3.167

2022-03-01

Ecology and Evolution

Abstract:Ecology and Evolution, Volume 12, Issue 3, February 2022.

ecology,evolutionary biology

What problem does this paper attempt to address?

BEEtag: A Low-Cost, Image-Based Tracking System for the Study of Animal Behavior and Locomotion

James D. Crall,Nick Gravish,Andrew M. Mountcastle,Stacey A. Combes

DOI: https://doi.org/10.1371/journal.pone.0136487

IF: 3.7

2015-09-02

PLoS ONE

Abstract:A fundamental challenge common to studies of animal movement, behavior, and ecology is the collection of high-quality datasets on spatial positions of animals as they change through space and time. Recent innovations in tracking technology have allowed researchers to collect large and highly accurate datasets on animal spatiotemporal position while vastly decreasing the time and cost of collecting such data. One technique that is of particular relevance to the study of behavioral ecology involves tracking visual tags that can be uniquely identified in separate images or movie frames. These tags can be located within images that are visually complex, making them particularly well suited for longitudinal studies of animal behavior and movement in naturalistic environments. While several software packages have been developed that use computer vision to identify visual tags, these software packages are either (a) not optimized for identification of single tags, which is generally of the most interest for biologists, or (b) suffer from licensing issues, and therefore their use in the study of animal behavior has been limited. Here, we present BEEtag, an open-source, image-based tracking system in Matlab that allows for unique identification of individual animals or anatomical markers. The primary advantages of this system are that it (a) independently identifies animals or marked points in each frame of a video, limiting error propagation, (b) performs well in images with complex backgrounds, and (c) is low-cost. To validate the use of this tracking system in animal behavior, we mark and track individual bumblebees (Bombus impatiens) and recover individual patterns of space use and activity within the nest. Finally, we discuss the advantages and limitations of this software package and its application to the study of animal movement, behavior, and ecology.

multidisciplinary sciences
Honey bee behaviours within the hive: Insights from long-term video analysis

Paul Siefert,Nastasya Buling,Bernd Grünewald

DOI: https://doi.org/10.1371/journal.pone.0247323

IF: 3.7

2021-03-17

PLoS ONE

Abstract:The combined behaviours of individuals within insect societies determine the survival and development of the colony. For the western honey bee ( Apis mellifera ), individual behaviours include nest building, foraging, storing and ripening food, nursing the brood, temperature regulation, hygiene and defence. However, the various behaviours inside the colony, especially within the cells, are hidden from sight, and until recently, were primarily described through texts and line drawings, which lack the dynamics of moving images. In this study, we provide a comprehensive source of online video material that offers a view of honey bee behaviour within comb cells, thereby providing a new mode of observation for the scientific community and the general public. We analysed long-term video recordings from longitudinally truncated cells, which allowed us to see sideways into the cells in the middle of a colony. Our qualitative study provides insight into worker behaviours, including the use of wax scales and existing nest material to remodel combs, storing pollen and nectar in cells, brood care and thermoregulation, and hygienic practices, such as cannibalism, grooming and surface cleaning. We reveal unique processes that have not been previously published, such as the rare mouth-to-mouth feeding by nurses to larvae as well as thermoregulation within cells containing the developing brood. With our unique video method, we are able to bring the processes of a fully functioning social insect colony into classrooms and homes, facilitating ecological awareness in modern times. We provide new details and images that will help scientists test their hypotheses on social behaviours. In addition, we encourage the non-commercial use of our material to educate beekeepers, the media and the public and, in turn, call attention to the general decline of insect biomass and diversity.

multidisciplinary sciences
Development of a new method to track multiple honey bees with complex behaviors on a flat laboratory arena

Toshifumi Kimura,Mizue Ohashi,Karl Crailsheim,Thomas Schmickl,Ryuichi Okada,Gerald Radspieler,Hidetoshi Ikeno

DOI: https://doi.org/10.1371/journal.pone.0084656

IF: 3.7

2014-01-20

PLoS ONE

Abstract:A computer program that tracks animal behavior, thereby revealing various features and mechanisms of social animals, is a powerful tool in ethological research. Because honeybee colonies are populated by thousands of bees, individuals co-exist in high physical densities and are difficult to track unless specifically tagged, which can affect behavior. In addition, honeybees react to light and recordings must be made under special red-light conditions, which the eyes of bees perceive as darkness. The resulting video images are scarcely distinguishable. We have developed a new algorithm, K-Track, for tracking numerous bees in a flat laboratory arena. Our program implements three main processes: (A) The object (bee's) region is detected by simple threshold processing on gray scale images, (B) Individuals are identified by size, shape and spatiotemporal positional changes, and (C) Centers of mass of identified individuals are connected through all movie frames to yield individual behavioral trajectories. The tracking performance of our software was evaluated on movies of mobile multi-artificial agents and of 16 bees walking around a circular arena. K-Track accurately traced the trajectories of both artificial agents and bees. In the latter case, K-track outperformed Ctrax, well-known software for tracking multiple animals. To investigate interaction events in detail, we manually identified five interaction categories; 'crossing', 'touching', 'passing', 'overlapping' and 'waiting', and examined the extent to which the models accurately identified these categories from bee's interactions. All 7 identified failures occurred near a wall at the outer edge of the arena. Finally, K-Track and Ctrax successfully tracked 77 and 60 of 84 recorded interactive events, respectively. K-Track identified multiple bees on a flat surface and tracked their speed changes and encounters with other bees, with good performance.
MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation

Luc Nicolas Stiemer,Andreas Thoma,Carsten Braun

DOI: https://doi.org/10.1371/journal.pone.0291415

IF: 3.7

2023-09-22

PLoS ONE

Abstract:This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come with intensive restrictions, such as the need for sufficient lighting, high contrast between the animal and background, absence of occlusion, significant user input, etc. Tracking flight paths of bumblebees in a social group is challenging. They suddenly adjust movements and change their appearance during different wing beat states while exhibiting significant similarities in their individual appearance. The MBT3D tracker, developed in this research, is an adaptation of an existing ant tracking algorithm for bumblebee tracking. It incorporates an offline trained appearance descriptor along with a Kalman Filter for appearance and motion matching. Different detector architectures for upstream detections (You Only Look Once (YOLOv5), Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and RetinaNet) are investigated in a comparative study to optimize performance. The detection models were trained on a dataset containing 11359 labeled bumblebee images. YOLOv5 reaches an Average Precision of AP = 53, 8%, Faster R-CNN achieves AP = 45, 3% and RetinaNet AP = 38, 4% on the bumblebee validation dataset, which consists of 1323 labeled bumblebee images. The tracker's appearance model is trained on 144 samples. The tracker (with Faster R-CNN detections) reaches a Multiple Object Tracking Accuracy MOTA = 93, 5% and a Multiple Object Tracking Precision MOTP = 75, 6% on a validation dataset containing 2000 images, competing with state-of-the-art computer vision methods. The framework allows reliable tracking of different bumblebees in the same video stream with rarely occurring identity switches (IDS). MBT3D has much lower IDS than other commonly used algorithms, with one of the lowest false positive rates, competing with state-of-the-art animal tracking algorithms. The developed framework reconstructs the 3-dimensional (3D) flight paths of the bumblebees by triangulation. It also handles and compares two alternative stereo camera pairs if desired.
IntelliBeeHive: An Automated Honey Bee, Pollen, and Varroa Destructor Monitoring System

Christian I. Narcia-Macias,Joselito Guardado,Jocell Rodriguez,Joanne Rampersad-Ammons,Erik Enriquez,Dong-Chul Kim

DOI: https://doi.org/10.48550/arXiv.2309.08955

2023-09-16

Abstract:Utilizing computer vision and the latest technological advancements, in this study, we developed a honey bee monitoring system that aims to enhance our understanding of Colony Collapse Disorder, honey bee behavior, population decline, and overall hive health. The system is positioned at the hive entrance providing real-time data, enabling beekeepers to closely monitor the hive's activity and health through an account-based website. Using machine learning, our monitoring system can accurately track honey bees, monitor pollen-gathering activity, and detect Varroa mites, all without causing any disruption to the honey bees. Moreover, we have ensured that the development of this monitoring system utilizes cost-effective technology, making it accessible to apiaries of various scales, including hobbyists, commercial beekeeping businesses, and researchers. The inference models used to detect honey bees, pollen, and mites are based on the YOLOv7-tiny architecture trained with our own data. The F1-score for honey bee model recognition is 0.95 and the precision and recall value is 0.981. For our pollen and mite object detection model F1-score is 0.95 and the precision and recall value is 0.821 for pollen and 0.996 for "mite". The overall performance of our IntelliBeeHive system demonstrates its effectiveness in monitoring the honey bee's activity, achieving an accuracy of 96.28 % in tracking and our pollen model achieved a F1-score of 0.831.

Computer Vision and Pattern Recognition
Individual honey bee tracking in a beehive environment using deep learning and Kalman filter

Panadda Kongsilp,Unchalisa Taetragool,Orawan Duangphakdee

DOI: https://doi.org/10.1038/s41598-023-44718-y

IF: 4.6

2024-01-12

Scientific Reports

Abstract:The honey bee is the most essential pollinator and a key contributor to the natural ecosystem. There are numerous ways for thousands of bees in a hive to communicate with one another. Individual trajectories and social interactions are thus complex behavioral features that can provide valuable information for an ecological study. To study honey bee behavior, the key challenges that have resulted from unreliable studies include complexity (high density of similar objects, small objects, and occlusion), the variety of background scenes, the dynamism of individual bee movements, and the similarity between the bee body and the background in the beehive. This study investigated the tracking of individual bees in a beehive environment using a deep learning approach and a Kalman filter. Detection of multiple bees and individual object segmentation were performed using Mask R-CNN with a ResNet-101 backbone network. Subsequently, the Kalman filter was employed for tracking multiple bees by tracking the body of each bee across a sequence of image frames. Three metrics were used to assess the proposed framework: mean average precision (mAP) for multiple-object detection and segmentation tasks, CLEAR MOT for multiple object tracking tasks, and MOTS for multiple object tracking and segmentation tasks. For CLEAR MOT and MOTS metrics, accuracy (MOTA and MOTSA) and precision (MOTP and MOTSP) are considered. By employing videos from a custom-designed observation beehive, recorded at a frame rate of 30 frames per second (fps) and utilizing a continuous frame rate of 10 fps as input data, our system displayed impressive performance. It yielded satisfactory outcomes for tasks involving segmentation and tracking of multiple instances of bee behavior. For the multiple-object segmentation task based on Mask R-CNN, we achieved a 0.85 mAP. For the multiple-object-tracking task with the Kalman filter, we achieved 77.48% MOTA, 79.79% MOTSP, and 79.56% recall. For the overall system for multiple-object tracking and segmentation tasks, we achieved 77.00% MOTSA, 75.60% MOTSP, and 80.30% recall.

multidisciplinary sciences
Analysis of antennal responses to motion stimuli in the honey bee by automated tracking using DeepLabCut

Kohno,H.,Kamata,S.,Kubo,T.

DOI: https://doi.org/10.1101/2023.04.24.538069

2023-04-25

bioRxiv

Abstract:Considering recent developments in gene manipulation methods for honey bees, establishing simple, robust, and indoor assay systems which can analyze behavioral components in detail is important for the rise of honey bee behavioral genetics. We focused on the antennal movements of the honey bee, which are used for not only multimodal sensory perception but also interactions between individuals. We developed an experimental system for analyzing the antennal responses (ARs) of the honey bee using DeepLabCut, a markerless posture-tracking tool using deep learning. The tracking of antennal movements during the presentation of vertical (downward and upward) motion stimuli using DeepLabCut successfully detected the ARs reported in the previous studies, where bees tilted their antennae in the direction opposite to the motion stimuli. In addition, we successfully detected ARs in response to horizontal (forward and backward) motion stimuli. An investigation of the developmental maturation of honey bee ARs showed that ARs to motion stimuli were not detected in bees immediately after emergence but became detectable through post-emergence development in an experience-independent manner. Furthermore, unsupervised clustering analysis using multidimensional data created by processing tracking data using DeepLabCut classified antennal movements into different clusters, suggesting that data-driven behavioral classification can apply to AR paradigms. These results reveal novel AR to visual stimuli and developmental maturation of ARs and suggest the efficacy of data-driven analysis for behavioral classification in behavioral studies of the honey bee.
Enhancing Pollinator Conservation towards Agriculture 4.0: Monitoring of Bees through Object Recognition

Ajay John Alex,Chloe M. Barnes,Pedro Machado,Isibor Ihianle,Gábor Markó,Martin Bencsik,Jordan J. Bird

2024-05-24

Abstract:In an era of rapid climate change and its adverse effects on food production, technological intervention to monitor pollinator conservation is of paramount importance for environmental monitoring and conservation for global food security. The survival of the human species depends on the conservation of pollinators. This article explores the use of Computer Vision and Object Recognition to autonomously track and report bee behaviour from images. A novel dataset of 9664 images containing bees is extracted from video streams and annotated with bounding boxes. With training, validation and testing sets (6722, 1915, and 997 images, respectively), the results of the COCO-based YOLO model fine-tuning approaches show that YOLOv5m is the most effective approach in terms of recognition accuracy. However, YOLOv5s was shown to be the most optimal for real-time bee detection with an average processing and inference time of 5.1ms per video frame at the cost of slightly lower ability. The trained model is then packaged within an explainable AI interface, which converts detection events into timestamped reports and charts, with the aim of facilitating use by non-technical users such as expert stakeholders from the apiculture industry towards informing responsible consumption and production.

Computer Vision and Pattern Recognition,Machine Learning
A versatile recording device for the analysis of continuous daily external activity in colonies of highly eusocial bees

Arthur Roque Justino,Klaus Hartfelder

DOI: https://doi.org/10.1007/s00359-024-01709-2

2024-06-20

Abstract:As pollinators, bees are key to maintaining the biodiversity of angiosperm plants, and for agriculture they provide a billion-dollar ecosystem service. But they also compete for resources (primarily nectar and pollen), especially the highly social bees that live in perennial colonies. So, how do they organize their daily temporal activities? Here, we present a versatile, low-cost device for the continuous, automatic recording and data analysis of the locomotor activity in the colony-entrance tube of highly eusocial bees. Consisting of an in-house built block containing an infrared detector, the passage of bees in the colony entrance tunnel is registered and automatically recorded in an Arduino environment, together with concomitant recordings of temperature and relative humidity. With a focus on the highly diverse Neotropical stingless bees (Meliponini), we obtained 10-day consecutive recordings for two colonies each of the species Melipona quadrifasciata and Frieseomelitta varia, and also for the honey bee. The Lomb-Scargle periodogram analysis identified a predominant circadian rhythmicity for all three species, but also indications of ultradian rhythms. For M. quadrifasciata, which is comparable in size to the honey bee, we found evidence for a possibly anticipatory activity already before sunrise. As all three species also presented activity at night in the colony entrance tube, this also raises questions about sleep organization in social insects. The cost and versatility of the device and the open-source options for data analysis make this an attractive system for conducting studies on circadian rhythms in social bees under natural conditions, complementing studies on flower visits by these important pollinators.
Automated monitoring of honey bees with barcodes and artificial intelligence reveals two distinct social networks from a single affiliative behavior

Tim Gernat,Tobias Jagla,Beryl M Jones,Martin Middendorf,Gene E Robinson

DOI: https://doi.org/10.1038/s41598-022-26825-4

2023-01-27

Abstract:Barcode-based tracking of individuals is revolutionizing animal behavior studies, but further progress hinges on whether in addition to determining an individual's location, specific behaviors can be identified and monitored. We achieve this goal using information from the barcodes to identify tightly bounded image regions that potentially show the behavior of interest. These image regions are then analyzed with convolutional neural networks to verify that the behavior occurred. When applied to a challenging test case, detecting social liquid transfer (trophallaxis) in the honey bee hive, this approach yielded a 67% higher sensitivity and an 11% lower error rate than the best detector for honey bee trophallaxis so far. We were furthermore able to automatically detect whether a bee donates or receives liquid, which previously required manual observations. By applying our trophallaxis detector to recordings from three honey bee colonies and performing simulations, we discovered that liquid exchanges among bees generate two distinct social networks with different transmission capabilities. Finally, we demonstrate that our approach generalizes to detecting other specific behaviors. We envision that its broad application will enable automatic, high-resolution behavioral studies that address a broad range of previously intractable questions in evolutionary biology, ethology, neuroscience, and molecular biology.
Integrating real-time data analysis into automatic tracking of social insects

Alessio Sclocco,Shirlyn Jia Yun Ong,Sai Yan Pyay Aung,Serafino Teseo

DOI: https://doi.org/10.1098/rsos.202033

2021-03-31

Abstract:Automatic video tracking has become a standard tool for investigating the social behaviour of insects. The recent integration of computer vision in tracking technologies will probably lead to fully automated behavioural pattern classification within the next few years. However, many current systems rely on offline data analysis and use computationally expensive techniques to track pre-recorded videos. To address this gap, we developed BACH (Behaviour Analysis maCHine), a software that performs video tracking of insect groups in real time. BACH uses object recognition via convolutional neural networks and identifies individually tagged insects via an existing matrix code recognition algorithm. We compared the tracking performances of BACH and a human observer (HO) across a series of short videos of ants moving in a two-dimensional arena. We found that BACH detected ant shapes only slightly worse than the HO. However, its matrix code-mediated identification of individual ants only attained human-comparable levels when ants moved relatively slowly, and fell when ants walked relatively fast. This happened because BACH had a relatively low efficiency in detecting matrix codes in blurry images of ants walking at high speeds. BACH needs to undergo hardware and software adjustments to overcome its present limits. Nevertheless, our study emphasizes the possibility of, and the need for, further integrating real-time data analysis into the study of animal behaviour. This will accelerate data generation, visualization and sharing, opening possibilities for conducting fully remote collaborative experiments.
Markerless tracking of an entire honey bee colony

Katarzyna Bozek,Laetitia Hebert,Yoann Portugal,Alexander S Mikheyev,Greg J Stephens

DOI: https://doi.org/10.1038/s41467-021-21769-1

2021-03-19

Abstract:From cells in tissue, to bird flocks, to human crowds, living systems display a stunning variety of collective behaviors. Yet quantifying such phenomena first requires tracking a significant fraction of the group members in natural conditions, a substantial and ongoing challenge. We present a comprehensive, computational method for tracking an entire colony of the honey bee Apis mellifera using high-resolution video on a natural honeycomb background. We adapt a convolutional neural network (CNN) segmentation architecture to automatically identify bee and brood cell positions, body orientations and within-cell states. We achieve high accuracy (~10% body width error in position, ~10° error in orientation, and true positive rate > 90%) and demonstrate months-long monitoring of sociometric colony fluctuations. These fluctuations include ~24 h cycles in the counted detections, negative correlation between bee and brood, and nightly enhancement of bees inside comb cells. We combine detected positions with visual features of organism-centered images to track individuals over time and through challenging occluding events, recovering ~79% of bee trajectories from five observation hives over 5 min timespans. The trajectories reveal important individual behaviors, including waggle dances and crawling inside comb cells. Our results provide opportunities for the quantitative study of collective bee behavior and for advancing tracking techniques of crowded systems.
The BumbleBox: An open-source platform for quantifying behavior in bumblebee colonies

August C Easton-Calabria,James D Crall

DOI: https://doi.org/10.1101/2024.11.07.622358

2024-11-08

Abstract:Bumblebees (Apini: Bombus) are important pollinators globally and an emerging model system for studying the ecology and evolution of social behavior and effects of environmental stressors on bees. Behavioral studies of bumblebees have conventionally relied on labor and time-intensive manual observations. While recent years have seen rapid advances in automated behavioral tracking in social insects, these tracking technologies are often expensive and require extensive programming experience, limiting accessibility and widespread adoption. Here we introduce the BumbleBox, an open-source system for automated tracking and behavioral quantification of individual bumblebees that can be built using low-cost consumer components and DIY fabrication (i.e., 3D-printing and laser-cutting). We provide an integrated pipeline for data collection and analysis, including nest arena design, software for automated collection of video data, and the quantification of individual behavior. The BumbleBox system is designed to be (a) accessible, requiring no prior experience with programming or hardware design to operate; (b) scalable, allowing long-term, automated tracking across many units in parallel at low-cost; and (c) modular, allowing for flexible adoption to unique applications in bumblebees and other systems. We validate the use of this system in a widespread bumblebee species (Bombus impatiens) that is both commercially and ecologically important. Finally, we highlight widespread potential applications in quantifying behavior and pollinator health in bumblebees and other social insects, including screening impacts of pesticides and other environmental stressors on social behavior.

Animal Behavior and Cognition
Unraveling Honey Bee's Waggle Dances in Natural Conditions with Video-Based Deep Learning

Sylvain Grison,Rajath Siddaganga,Shrihari Hedge,James Burridge,Pieter M. Blok,Smitha Krishnan,Axel Brockmann,Wei Guo

DOI: https://doi.org/10.1101/2024.11.16.623982

2024-11-18

Abstract:1- Wild and managed honey bees, crucial pollinators for both agriculture and natural ecosystems, face challenges due to industrial agriculture and urbanization. Understanding how bee colonies utilize the landscape for foraging is essential for managing human-bee conflicts and protecting these pollinators to sustain their vital pollination services. To understand how the bees utilize their surroundings, researchers often decode bee waggle dances, which honey bee workers use to communicate navigational information of desirable food and nesting sites to their nest mates. This process is carried out manually, which is time-consuming, prone to human error and requires specialized skills. 2- We address this problem by introducing a novel deep learning-based pipeline that automatically detects and measures waggle runs, the core movement of the waggle dance, under natural recording conditions for the first time. We combined the capabilities of the action detector YOWOv2 and the DeepSORT tracking method, with the Principal Component Analysis to extract dancing bee bounding boxes and the angles and durations within waggle runs. 3- The presented pipeline works fully automatically with videos taken from wild Apis dorsata colonies in its natural environment, and can be used for any honey bee species. Comparison of our pipeline with analyses made by human experts revealed that our procedure was able to detect 93\% of waggle runs on the testing dataset, with a run duration Root Mean Squared Error (RMSE) of less than a second, and a run angle RMSE of 0.14 radians. We also assessed the generalizability of our pipeline to previously unseen recording conditions, successfully detecting 50% of waggle runs performed by Apis mellifera bees from a colony managed in Tokyo, Japan. In parallel, we discovered the most appropriate values of the model's hyperparameters for this task. 4- Our study demonstrates that a deep learning-based pipeline can successfully and automatically analyze the waggle runs of Apis dorsata in natural conditions and generalize to other bee species. This approach enables precise measurement of direction and duration, enabling the study of bee foraging behavior on an unprecedented scale compared to traditional manual methods contributing to preserving biodiversity and ecosystem services.

Ecology
Autonomous tracking of honey bee behaviors over long-term periods with cooperating robots

Jiří Ulrich,Martin Stefanec,Fatemeh Rekabi-Bana,Laurenz Alexander Fedotoff,Tomáš Rouček,Bilal Yağız Gündeğer,Mahmood Saadat,Jan Blaha,Jiří Janota,Daniel Nicolas Hofstadler,Kristina Žampachů,Erhan Ege Keyvan,Babür Erdem,Erol Şahin,Hande Alemdar,Ali Emre Turgut,Farshad Arvin,Thomas Schmickl,Tomáš Krajník

DOI: https://doi.org/10.1126/scirobotics.adn6848

IF: 25

2024-10-18

Science Robotics

Abstract:Digital and mechatronic methods, paired with artificial intelligence and machine learning, are transformative technologies in behavioral science and biology. The central element of the most important pollinator species—honey bees—is the colony's queen. Because honey bee self-regulation is complex and studying queens in their natural colony context is difficult, the behavioral strategies of these organisms have not been widely studied. We created an autonomous robotic observation and behavioral analysis system aimed at continuous observation of the queen and her interactions with worker bees and comb cells, generating behavioral datasets of exceptional length and quality. Key behavioral metrics of the queen and her social embedding within the colony were gathered using our robotic system. Data were collected continuously for 24 hours a day over a period of 30 days, demonstrating our system's capability to extract key behavioral metrics at microscopic, mesoscopic, and macroscopic system levels. Additionally, interactions among the queen, worker bees, and brood were observed and quantified. Long-term continuous observations performed by the robot yielded large amounts of high-definition video data that are beyond the observation capabilities of humans or stationary cameras. Our robotic system can enable a deeper understanding of the innermost mechanisms of honey bees' swarm-intelligent self-regulation. Moreover, it offers the possibility to study other social insect colonies, biocoenoses, and ecosystems in an automated manner. Social insects are keystone species in all terrestrial ecosystems; thus, developing a better understanding of their behaviors will be invaluable for the protection and even the restoration of our fragile ecosystems globally.

robotics
Intelligent Beehive Monitoring System Based on Internet of Things and Colony State Analysis

Yiyao Zheng,Xiaoyan Cao,Shaocong Xu,Shihui Guo,Rencai Huang,Yingjiao Li,Yijie Chen,Liulin Yang,Xiaoyu Cao,Zainura Idrus,Hongting Sun

DOI: https://doi.org/10.1016/j.atech.2024.100584

2024-01-01

Smart Agricultural Technology

Abstract:Bees play a crucial role in terrestrial ecosystems. However, beekeepers are unable to monitor the state of beehives (bees and environment) all the time, which often results in bees escaping or even dying. Currently, some researchers provided the scheme of intelligent beehive monitoring system equipped with the Internet of Things (IoT), There remain two challenges: accurately monitor the environmental status around the hive and accurately track and monitor bees in real time. With the development of the IoT and computer vision algorithms, we hope to provide an automated and efficient system to meet the above challenges. In this paper, we proposed a hive monitoring system, and build a visualization module in the cloud to monitor the activity of bee colonies and the environmental dynamic changes. (1) We proposed a multi-bee tracking algorithm to solve the problem of monitoring bees at the door of the hive; (2) we constructed a dataset containing various complex scenes, named BEE22, for training and testing the performance of our algorithm; (3) we designed a bee counting rule, based on results of multi-bee tracking algorithm, to reasonably count the bees entering or leaving the beehive; (4) we have deployed multiple sensors around(center, margin, door, and environment) the hive to accurately reflect the changes in the environment around the hive. Experimental results demonstrate the effectiveness and excellence of our system. In particular, the tracking performance of the multi-bee tracking algorithm reaches 83.5 % +/- 0.7 % Multiple Object Tracking Accuracy (MOTA) and 77.3 %+/- 0.2 % Multiple Object Tracking Precision (MOTP), speeds up to 16 frames per second, compared with other algorithms, MOTA and Identity F1 Score (IDF1) are improved by 5.4 % and 8.2 % respectively. Moreover, our counting algorithm also achieved excellent results, with root mean square error (RMSE) of 1.3 +/- 0.1, 0.2 +/- 0.0, and 1.6 +/- 0.1 in counting the number of bees current, entry, and out scene in an episode, respectively. After that, the system will be deployed and monitored for a long time in the actual scenario, it was found that the activity of bees decreased significantly under heavy rainfall conditions. Additionally, the activity of the bee colony will also increase accordingly, when the amplitude is 500 dB to 2000 dB, the temperature of the center of the beehive is 25 degrees C to 37 degrees C, and the humidity is 48 % to 67 %. In summary, our system can provide valuable information for bee farmers to make control decisions on hives.
Automated Beehive Acoustics Monitoring: A Comprehensive Review of the Literature and Recommendations for Future Work

Mahsa Abdollahi,Pierre Giovenazzo,Tiago H. Falk

DOI: https://doi.org/10.3390/app12083920

2022-04-13

Applied Sciences

Abstract:Bees play an important role in agriculture and ecology, and their pollination efficiency is essential to the economic profitability of farms. The drastic decrease in bee populations witnessed over the last decade has attracted great attention to automated remote beehive monitoring research, with beehive acoustics analysis emerging as a prominent field. In this paper, we review the existing literature on bee acoustics analysis and report on the articles published between January 2012 and December 2021. Five categories are explored in further detail, including the origin of the articles, their study goal, experimental setup, audio analysis methodology, and reproducibility. Highlights and limitations in each of these categories are presented and discussed. We conclude with a set of recommendations for future studies, with suggestions ranging from bee species characterization, to recording and testing setup descriptions, to making data and codes available to help advance this new multidisciplinary field.
Bee Cluster 3D: A system to monitor the temperature in a hive over time

Olivier Besson,Hervé Mugnier,Aurélien Neveux,Gaëlle Rey,Sully Vitry

DOI: https://doi.org/10.48550/arXiv.2101.03125

2021-01-09

Abstract:A new system, Bee Cluster 3D, allowing the study of the time evolution of the 3D temperature distribution in a bee hive is presented. This system can be used to evaluate the cluster size and the location of the queen during winter. In summer, the device can be used to quantify the size of the brood nest and the breeding activity of the queen. The system does not disturb the activity of the colony and can be used on any hive. This electronic system was developed to be non-intrusive, miniaturized, and energy autonomous.

Quantitative Methods
Flower visitation through the lens: Exploring the foraging behaviour of with a computer vision-based application

Zsófia Varga-Szilay,Gergely Szövényi,Gábor Pozsgai

DOI: https://doi.org/10.1101/2024.07.10.602888

2024-07-15

Abstract:To understand the processes behind pollinator declines, and thus to maintain pollination efficiency, we also have to understand fundamental drivers influencing pollinator behaviour. In this study, we aim to explore the foraging behaviour of wild bumblebees, recognizing its importance from economic and conservation perspectives. We recorded on , , and patches in five-minute-long slots in urban areas of Terceira (Azores, Portugal). For the automated bumblebee detection, we created computer vision models based on a deep learning algorithm, with custom datasets. We achieved high F1 scores of 0.88 for and , and 0.95 for , indicating accurate bumblebee detection. We found that flower cover per cent, but not plant species, influenced the attractiveness of flower patches, with a significant positive effect. There were no differences between plant species in the attractiveness of the flower heads. The handling time was longer on the large-headed than those on the smaller-headed and . However, our result did not indicate significant differences in the time bumblebees spent on flowers among the three plant species. Here, we also justify computer vision-based analysis as a reliable tool for studying pollinator behavioural ecology.

Animal Behavior and Cognition
System design for inferring colony-level pollination activity through miniature bee-mounted sensors

Haron M. Abdel-Raziq,Daniel M. Palmer,Phoebe A. Koenig,Alyosha C. Molnar,Kirstin H. Petersen

DOI: https://doi.org/10.1038/s41598-021-82537-1

IF: 4.6

2021-02-19

Scientific Reports

Abstract:Abstract In digital agriculture, large-scale data acquisition and analysis can improve farm management by allowing growers to constantly monitor the state of a field. Deploying large autonomous robot teams to navigate and monitor cluttered environments, however, is difficult and costly. Here, we present methods that would allow us to leverage managed colonies of honey bees equipped with miniature flight recorders to monitor orchard pollination activity. Tracking honey bee flights can inform estimates of crop pollination, allowing growers to improve yield and resource allocation. Honey bees are adept at maneuvering complex environments and collectively pool information about nectar and pollen sources through thousands of daily flights. Additionally, colonies are present in orchards before and during bloom for many crops, as growers often rent hives to ensure successful pollination. We characterize existing Angle-Sensitive Pixels (ASPs) for use in flight recorders and calculate memory and resolution trade-offs. We further integrate ASP data into a colony foraging simulator and show how large numbers of flights refine system accuracy, using methods from robotic mapping literature. Our results indicate promising potential for such agricultural monitoring, where we leverage the superiority of social insects to sense the physical world, while providing data acquisition on par with explicitly engineered systems.

multidisciplinary sciences

Bee Tracker—an open‐source machine learning‐based video analysis software for the assessment of nesting and foraging performance of cavity‐nesting solitary bees

BEEtag: A Low-Cost, Image-Based Tracking System for the Study of Animal Behavior and Locomotion

Honey bee behaviours within the hive: Insights from long-term video analysis

Development of a new method to track multiple honey bees with complex behaviors on a flat laboratory arena

MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation

IntelliBeeHive: An Automated Honey Bee, Pollen, and Varroa Destructor Monitoring System

Individual honey bee tracking in a beehive environment using deep learning and Kalman filter

Analysis of antennal responses to motion stimuli in the honey bee by automated tracking using DeepLabCut

Enhancing Pollinator Conservation towards Agriculture 4.0: Monitoring of Bees through Object Recognition

A versatile recording device for the analysis of continuous daily external activity in colonies of highly eusocial bees

Automated monitoring of honey bees with barcodes and artificial intelligence reveals two distinct social networks from a single affiliative behavior

Integrating real-time data analysis into automatic tracking of social insects

Markerless tracking of an entire honey bee colony

The BumbleBox: An open-source platform for quantifying behavior in bumblebee colonies

Unraveling Honey Bee's Waggle Dances in Natural Conditions with Video-Based Deep Learning

Autonomous tracking of honey bee behaviors over long-term periods with cooperating robots

Intelligent Beehive Monitoring System Based on Internet of Things and Colony State Analysis

Automated Beehive Acoustics Monitoring: A Comprehensive Review of the Literature and Recommendations for Future Work

Bee Cluster 3D: A system to monitor the temperature in a hive over time

Flower visitation through the lens: Exploring the foraging behaviour of with a computer vision-based application

System design for inferring colony-level pollination activity through miniature bee-mounted sensors