Abstract:Studying animal behaviour allows us to understand how different species and individuals navigate their physical and social worlds. Video coding of behaviour is considered a gold standard: allowing researchers to extract rich nuanced behavioural datasets, validate their reliability, and for research to be replicated. However, in practice, videos are only useful if data can be efficiently extracted. Manually locating relevant footage in 10,000 s of hours is extremely time-consuming, as is the manual coding of animal behaviour, which requires extensive training to achieve reliability. Machine learning approaches are used to automate the recognition of patterns within data, considerably reducing the time taken to extract data and improving reliability. However, tracking visual information to recognise nuanced behaviour is a challenging problem and, to date, the tracking and pose-estimation tools used to detect behaviour are typically applied where the visual environment is highly controlled. Animal behaviour researchers are interested in applying these tools to the study of wild animals, but it is not clear to what extent doing so is currently possible, or which tools are most suited to particular problems. To address this gap in knowledge, we describe the new tools available in this rapidly evolving landscape, suggest guidance for tool selection, provide a worked demonstration of the use of machine learning to track movement in video data of wild apes, and make our base models available for use. We use a pose-estimation tool, DeepLabCut, to demonstrate successful training of two pilot models of an extremely challenging pose estimate and tracking problem: multi-animal wild forest-living chimpanzees and bonobos across behavioural contexts from hand-held video footage. With DeepWild we show that, without requiring specific expertise in machine learning, pose estimation and movement tracking of free-living wild primates in visually complex environments is an attainable goal for behavioural researchers.

MonkeyTrail: A Scalable Video-Based Method for Tracking Macaque Movement Trajectory in Daily Living Cages.

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture

MacqD - A Deep Learning-based Model for Automatic Detection of Socially-housed Laboratory Macaques

A Real-time Multi-Subject Three Dimensional Pose Tracking System for Analyzing Social Behaviors of Non-human Primates

MonkeyPosekit: Automated Markerless 2D Pose Estimation of Monkey

AlphaChimp: Tracking and Behavior Recognition of Chimpanzees

MacaquePose: A Novel “In the Wild” Macaque Monkey Pose Dataset for Markerless Motion Capture

AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool

BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos

DeepWild: Application of the pose estimation tool DeepLabCut for behaviour tracking in wild chimpanzees and bonobos

Markerless tracking of user-defined features with deep learning

vmTracking: Virtual Markers Overcome Occlusion and Crowding in Multi-Animal Pose Tracking

Deep learning-based activity recognition and fine motor identification using 2D skeletons of cynomolgus monkeys

Automated Tracking of Primate Behavior

Telemetric recordings of single neuron activity and visual scenes in monkeys walking in an open field.

Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking

Automated video-based heart rate tracking for the anesthetized and behaving monkey

Automatic identification of stone-handling behaviour in Japanese macaques using LabGym artificial intelligence