Abstract:Developments in automated animal behavioural analysis software are increasing the efficiency of data collection and improving the standardization of behavioural measurements. There are now several open-source tools for tracking laboratory animals, but often these are only accurate under limited conditions (e.g. uniform lighting and background, uncluttered scenes, unobstructed focal animal). Tracking fish presents a particular challenge for these tools because movement at the water's surface introduces significant noise. Partial occlusion of the focal animal can also be troublesome, particularly when tracking the whole organism. But identifying the position of an animal is only part of the task -- analysing the movement of the animal relative to their environment and experimental context is often what provides information about their behaviour. Therefore, the automated detection of physical objects and boundaries would also be beneficial, but this feature is not commonly incorporated into existing tracking software. Here we describe a video processing method that uses a range of computer vision algorithms (e.g. object detector and tracker, optical flow, parallel plane homology) and computational geometry techniques (e.g. Voronoi tessellation) to analyse the movement behaviour of fish in response to experimental stimuli. A behavioural experiment, which involved tracking a fish's trajectory through a field of obstacles, motivated our development of a set of tools that: (1) measure an animal's trajectory, (2) record obstacle position, and (3) detect when the fish passed through 'virtual gates' between adjacent obstacles and/or the aquarium wall. We have introduced a novel Detect+Track approach that significantly enhances the accuracy and robustness of animal tracking, overcoming some of the limitations of existing tools and providing a more reliable solution for complex experimental conditions. Our workflow is divided into several discrete steps, and provides a set of modular software building blocks that can be adapted to analyse other experimental designs. A detailed tutorial is provided, together with all the data and code required to reproduce our results.

Watching Swarm Dynamics from Above: A Framework for Advanced Object Tracking in Drone Videos

Closed-Loop Tracking-by-Detection for ROV-Based Multiple Fish Tracking

Flow-Guided Single Object Tracking Framework in UAV Aerial Video

An Open-Source Platform For Underwater Image And Video Analytics

An Autonomous Drone Swarm for Detecting and Tracking Anomalies among Dense Vegetation

Animal Tracking within a Formation of Drones

Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms

Detect+Track: Robust and flexible software tools for improved tracking and behavioural analysis of fish

Automatic 3D Tracking System for Large Swarm of Moving Objects

Vision-based Drone Flocking in Outdoor Environments

Swarm behavior tracking based on a deep vision algorithm

Tracking the 3D Position and Orientation of Flying Swarms with Learned Kinematic Pattern Using LSTM Network.

Fish Tracking and Segmentation From Stereo Videos on the Wild Sea Surface for Electronic Monitoring of Rail Fishing

Automated 3D trajectory measuring of large numbers of moving particles

Multi-camera Realtime 3D Tracking of Multiple Flying Animals

Automatically Detect and Track Multiple Fish Swimming in Shallow Water with Frequent Occlusion.

FishTrack23: An Ensemble Underwater Dataset for Multi-Object Tracking

High-resolution, non-invasive animal tracking and reconstruction of local environment in aquatic ecosystems

Acquiring 3D motion trajectories of large numbers of swarming animals

Automatic detection of fish and tracking of movement for ecology

Online 3D Behavioral Tracking of Aquatic Model Organism with a Dual-Camera System