Abstract:The honey bee is the most essential pollinator and a key contributor to the natural ecosystem. There are numerous ways for thousands of bees in a hive to communicate with one another. Individual trajectories and social interactions are thus complex behavioral features that can provide valuable information for an ecological study. To study honey bee behavior, the key challenges that have resulted from unreliable studies include complexity (high density of similar objects, small objects, and occlusion), the variety of background scenes, the dynamism of individual bee movements, and the similarity between the bee body and the background in the beehive. This study investigated the tracking of individual bees in a beehive environment using a deep learning approach and a Kalman filter. Detection of multiple bees and individual object segmentation were performed using Mask R-CNN with a ResNet-101 backbone network. Subsequently, the Kalman filter was employed for tracking multiple bees by tracking the body of each bee across a sequence of image frames. Three metrics were used to assess the proposed framework: mean average precision (mAP) for multiple-object detection and segmentation tasks, CLEAR MOT for multiple object tracking tasks, and MOTS for multiple object tracking and segmentation tasks. For CLEAR MOT and MOTS metrics, accuracy (MOTA and MOTSA) and precision (MOTP and MOTSP) are considered. By employing videos from a custom-designed observation beehive, recorded at a frame rate of 30 frames per second (fps) and utilizing a continuous frame rate of 10 fps as input data, our system displayed impressive performance. It yielded satisfactory outcomes for tasks involving segmentation and tracking of multiple instances of bee behavior. For the multiple-object segmentation task based on Mask R-CNN, we achieved a 0.85 mAP. For the multiple-object-tracking task with the Kalman filter, we achieved 77.48% MOTA, 79.79% MOTSP, and 79.56% recall. For the overall system for multiple-object tracking and segmentation tasks, we achieved 77.00% MOTSA, 75.60% MOTSP, and 80.30% recall.

Automated monitoring of honey bees with barcodes and artificial intelligence reveals two distinct social networks from a single affiliative behavior

Automated monitoring of behavior reveals bursty interaction patterns and rapid spreading dynamics in honeybee social networks

Markerless tracking of an entire honey bee colony

Autonomous tracking of honey bee behaviors over long-term periods with cooperating robots

Individual honey bee tracking in a beehive environment using deep learning and Kalman filter

Development of a new method to track multiple honey bees with complex behaviors on a flat laboratory arena

Tracking all members of a honey bee colony over their lifetime

Molecular analyses of individual variation in honey bee sociability

Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms

IntelliBeeHive: An Automated Honey Bee, Pollen, and Varroa Destructor Monitoring System

Biohybrid Superorganisms—On the Design of a Robotic System for Thermal Interactions With Honeybee Colonies

A Computational Topology-based Spatiotemporal Analysis Technique for Honeybee Aggregation

A robotic honeycomb for interaction with a honeybee colony

Honey bee behaviours within the hive: Insights from long-term video analysis

Development of Behavior Monitoring System for Honeybees in Hive Using RFID Sensors and Image Processing

System design for inferring colony-level pollination activity through miniature bee-mounted sensors

Small Object Tracking in High Density Crowd Scenes

Analysis of antennal responses to motion stimuli in the honey bee by automated tracking using DeepLabCut

Integrating real-time data analysis into automatic tracking of social insects

BEEtag: A Low-Cost, Image-Based Tracking System for the Study of Animal Behavior and Locomotion

Pixel personality for dense object tracking in a 2D honeybee hive