Abstract:ABSTRACT Robust quantification of animal behavior is fundamental in experimental neuroscience research. Systems providing automated behavioral assessment are an important alternative to manual measurements avoiding problems such as human bias, low reproducibility and high cost. Integrating these tools with closed-loop control systems creates conditions to correlate environment and behavioral expressions effectively, and ultimately explain the neural foundations of behavior. We present an integrated solution for automated behavioral analysis of rodents using deep learning networks on video streams acquired from a depth-sensing camera. The use of depth sensors has notable advantages: tracking/classification performance is improved and independent of animals’ coat color, and videos can be recorded in dark conditions without affecting animals’ natural behavior. Convolutional and recurrent layers were combined in deep network architectures, and both spatial and temporal representations were successfully learned for a 4- classes behavior classification task (standstill, walking, rearing and grooming). Integration with Arduino microcontrollers creates an easy-to-use control platform providing low-latency feedback signals based on the deep learning automatic classification of animal behavior. The complete system, combining depth-sensor camera, computer, and Arduino microcontroller, allows simple mapping of input-output control signals using the animal’s current behavior and position. For example, a feeder can be controlled not by pressing a lever but by the animal behavior itself. An integrated graphical user interface completes a user-friendly and cost-effective solution for animal tracking and behavior classification. This open-software/open-hardware platform can boost the development of customized protocols for automated behavioral research, and support ever more sophisticated, reliable and reproducible behavioral neuroscience experiments.

Markerless tracking of user-defined features with deep learning

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Markerless Mouse Tracking for Social Experiments

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

vmTracking: Virtual Markers Overcome Occlusion and Crowding in Multi-Animal Pose Tracking

Enhanced Multi-Object Tracking: Inferring Motion States of Tracked Objects

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

High-resolution in vivo kinematic tracking with injectable fluorescent nanoparticles

Behavioral tracking gets real

MouseVenue3D: A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

Markerless Rat Behavior Quantification With Cascade Neural Network

PMotion: An advanced markerless pose estimation approach based on novel deep learning framework used to reveal neurobehavior

Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking

Anatomical-Marker-Driven 3D Markerless Human Motion Capture

AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool

Real-Time Closed-Loop Feedback in Behavioral Time Scales Using DeepLabCut

KineWheel-DeepLabCut Automated Paw Annotation Using Alternating Stroboscopic UV and White Light Illumination

Deep learning-based system for real-time behavior recognition and closed-loop control of behavioral mazes using depth sensing