Abstract:This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come with intensive restrictions, such as the need for sufficient lighting, high contrast between the animal and background, absence of occlusion, significant user input, etc. Tracking flight paths of bumblebees in a social group is challenging. They suddenly adjust movements and change their appearance during different wing beat states while exhibiting significant similarities in their individual appearance. The MBT3D tracker, developed in this research, is an adaptation of an existing ant tracking algorithm for bumblebee tracking. It incorporates an offline trained appearance descriptor along with a Kalman Filter for appearance and motion matching. Different detector architectures for upstream detections (You Only Look Once (YOLOv5), Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and RetinaNet) are investigated in a comparative study to optimize performance. The detection models were trained on a dataset containing 11359 labeled bumblebee images. YOLOv5 reaches an Average Precision of AP = 53, 8%, Faster R-CNN achieves AP = 45, 3% and RetinaNet AP = 38, 4% on the bumblebee validation dataset, which consists of 1323 labeled bumblebee images. The tracker's appearance model is trained on 144 samples. The tracker (with Faster R-CNN detections) reaches a Multiple Object Tracking Accuracy MOTA = 93, 5% and a Multiple Object Tracking Precision MOTP = 75, 6% on a validation dataset containing 2000 images, competing with state-of-the-art computer vision methods. The framework allows reliable tracking of different bumblebees in the same video stream with rarely occurring identity switches (IDS). MBT3D has much lower IDS than other commonly used algorithms, with one of the lowest false positive rates, competing with state-of-the-art animal tracking algorithms. The developed framework reconstructs the 3-dimensional (3D) flight paths of the bumblebees by triangulation. It also handles and compares two alternative stereo camera pairs if desired.

Pixel personality for dense object tracking in a 2D honeybee hive

Markerless tracking of an entire honey bee colony

Development of a new method to track multiple honey bees with complex behaviors on a flat laboratory arena

Individual honey bee tracking in a beehive environment using deep learning and Kalman filter

Tracking all members of a honey bee colony over their lifetime

Small Object Tracking in High Density Crowd Scenes

MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation

Automated 3D trajectory measuring of large numbers of moving particles

Autonomous tracking of honey bee behaviors over long-term periods with cooperating robots

BEEtag: A Low-Cost, Image-Based Tracking System for the Study of Animal Behavior and Locomotion

Smooth 3D Trajectory Segmentation for Flying Insects

Automated monitoring of honey bees with barcodes and artificial intelligence reveals two distinct social networks from a single affiliative behavior

Swarm behavior tracking based on a deep vision algorithm

Shape-and-behavior encoded tracking of bee dances

Interactive Framework for Insect Tracking with Active Learning

Multi-object Tracking Based on Particle Probability Hypothesis Density Tracker in Microscopic Video

Watching Swarm Dynamics from Above: A Framework for Advanced Object Tracking in Drone Videos

Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms

Automatic 3D Tracking System for Large Swarm of Moving Objects

Multi-appearance segmentation and extended 0-1 programming for dense small object tracking

DenseTrack: Drone-based Crowd Tracking via Density-aware Motion-appearance Synergy