Abstract:Studying animal behaviour allows us to understand how different species and individuals navigate their physical and social worlds. Video coding of behaviour is considered a gold standard: allowing researchers to extract rich nuanced behavioural datasets, validate their reliability, and for research to be replicated. However, in practice, videos are only useful if data can be efficiently extracted. Manually locating relevant footage in 10,000 s of hours is extremely time-consuming, as is the manual coding of animal behaviour, which requires extensive training to achieve reliability. Machine learning approaches are used to automate the recognition of patterns within data, considerably reducing the time taken to extract data and improving reliability. However, tracking visual information to recognise nuanced behaviour is a challenging problem and, to date, the tracking and pose-estimation tools used to detect behaviour are typically applied where the visual environment is highly controlled. Animal behaviour researchers are interested in applying these tools to the study of wild animals, but it is not clear to what extent doing so is currently possible, or which tools are most suited to particular problems. To address this gap in knowledge, we describe the new tools available in this rapidly evolving landscape, suggest guidance for tool selection, provide a worked demonstration of the use of machine learning to track movement in video data of wild apes, and make our base models available for use. We use a pose-estimation tool, DeepLabCut, to demonstrate successful training of two pilot models of an extremely challenging pose estimate and tracking problem: multi-animal wild forest-living chimpanzees and bonobos across behavioural contexts from hand-held video footage. With DeepWild we show that, without requiring specific expertise in machine learning, pose estimation and movement tracking of free-living wild primates in visually complex environments is an attainable goal for behavioural researchers.

DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Markerless tracking of user-defined features with deep learning

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Multi-animal pose estimation and tracking with DeepLabCut

Multi-animal pose estimation, identification and tracking with DeepLabCut

Deep-Learning-Based Markerless Pose Estimation Systems in Gait Analysis: DeepLabCut Custom Training and the Refinement Function

PMotion: An advanced markerless pose estimation approach based on novel deep learning framework used to reveal neurobehavior

Validation of DeepLabCut as a tool for markerless 3D pose estimation

DeepLabStream enables closed-loop behavioral experiments using deep learning-based markerless, real-time posture detection

Anatomical-Marker-Driven 3D Markerless Human Motion Capture

A Review of Deep Learning Techniques for Markerless Human Motion on Synthetic Datasets

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

Using DeepLabCut for tracking body landmarks in videos of children with dyskinetic cerebral palsy: a working methodology

Markerless Mouse Tracking for Social Experiments

MonkeyPosekit: Automated Markerless 2D Pose Estimation of Monkey

A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives

Deep learning tools for the measurement of animal behavior in neuroscience

Towards Accurate Markerless Human Shape and Pose Estimation over Time

KineWheel-DeepLabCut Automated Paw Annotation Using Alternating Stroboscopic UV and White Light Illumination

DeepWild: Application of the pose estimation tool DeepLabCut for behaviour tracking in wild chimpanzees and bonobos