Abstract:Collective animal behavior occurs in groups and swarms at almost every biological scale, from single-celled organisms to the largest animals on Earth. The intriguing mysteries behind these group behaviors have attracted many scholars, and while it is known that models can reproduce qualitative features of such complex behaviors, this requires data from real animals to demonstrate, and obtaining data on the exact features of these groups is tricky. In this paper, we propose the Hidden Markov Unscented Tracker (HMUT), which combines the state prediction capability of HMM and the high-precision nonlinear processing capability of UKF. This prediction-driven tracking mechanism enables HMUT to quickly adjust tracking strategies when facing sudden changes in target motion direction or rapid changes in speed, reducing the risk of tracking loss. Videos of fruit fly swarm movement in an enclosed environment are captured using stereo cameras. For the captured fruit fly images, the thresholded AKAZE algorithm is first used to detect the positions of individual fruit flies in the images, and the motion of the fruit flies is modeled using a multidimensional hidden Markov model (HMM). Tracking is then performed using the Unscented Kalman Filter algorithm to obtain the flight trajectories of the fruit flies in two camera views. Finally, 3D reconstruction of the trajectories in both views is achieved through polar coordinate constraints, resulting in 3D motion data of the fruit flies. Additionally, the efficiency and accuracy of the proposed algorithm are evaluated by simulating fruit fly swarm movement using the Boids algorithm. Finally, based on the tracked fruit fly flight data, behavioral characteristics of the fruit flies are analyzed from two perspectives. The first is a statistical analysis of the differences between the two behaviors. The second dimension involves clustering trajectory similarity using the DTW method based on fruit fly flight trajectories, further analyzing the similarity within clusters and differences between clusters.

An Unsupervised Method for Quantifying the Behavior of Interacting Individuals

Automated Monitoring and Analysis of Social Behavior in Drosophila

Probabilistic modeling reveals coordinated social interaction states and their multisensory bases

Comparative analysis of tracking and behavioral patterns between wild-type and genetically modified fruit flies using computer vision and statistical methods

A Novel Method For Tracking Individuals Of Fruit Fly Swarms Flying In A Laboratory Flight Arena

Behavioral signatures of structured feature detection during courtship in Drosophila

A Simple Computer Vision Pipeline Reveals the Effects of Isolation on Social Interaction Dynamics in Drosophila

Quantifying social roles in multi-animal videos using subject-aware deep-learning

Modelling collective behavior in groups of mice housed under semi-naturalistic conditions

Mapping the stereotyped behaviour of freely-moving fruit flies

Mapping the landscape of social behavior

An automatic system for recognizing fly courtship patterns via an image processing method

Capturing continuous, long timescale behavioral changes in $\textit{Drosophila melanogaster}$ postural data

Leg-tracking and Automated Behavioural Classification in Drosophila

Misconceptions about quantifying animal encounter and interaction processes

DVT: a high-throughput analysis pipeline for locomotion and social behavior in adult Drosophila melanogaster

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning

Eavesdropping and the audience effect in fighting fruit flies reveal feedback between social information use and production

Robust Tracking and Behavioral Modeling of Movements of Biological Collectives from Ordinary Video Recordings

The Fly Disco: Hardware and software for optogenetics and fine-grained fly behavior analysis

Leg-tracking and automated behavioral classification in Drosophila