Abstract:Automated behavior quantification in socially interacting animals requires accurate tracking. While many methods have been very successful and highly generalizable to different settings, issues of mistaken identities and lost information on key anatomical features are common, although they can be alleviated by increased human effort in training or post-processing. We propose a markerless video-based tool to simultaneously track two interacting mice of the same appearance in controlled settings for quantifying behaviors such as different types of sniffing, touching, and locomotion to improve tracking accuracy under these settings without increased human effort. It incorporates conventional handcrafted tracking and deep-learning-based techniques. The tool is trained on a small number of manually annotated images from a basic experimental setup and outputs body masks and coordinates of the snout and tail-base for each mouse. The method was tested on several commonly used experimental conditions including bedding in the cage and fiberoptic or headstage implants on the mice. Results obtained without any human corrections after the automated analysis showed a near elimination of identities switches and a ~15% improvement in tracking accuracy over pure deep-learning-based pose estimation tracking approaches. Our approach can be optionally ensembled with such techniques for further improvement. Finally, we demonstrated an application of this approach in studies of social behavior of mice by quantifying and comparing interactions between pairs of mice in which some lack olfaction. Together, these results suggest that our approach could be valuable for studying group behaviors in rodents, such as social interactions.

Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model

Multi-View Mouse Social Behaviour Recognition With Deep Graphic Model

Context-Aware Mouse Behavior Recognition Using Hidden Markov Models

Behavior Recognition in Mouse Videos Using Contextual Features Encoded by Spatial-temporal Stacked Fisher Vectors.

Social Behavior Recognition in Mouse Video Using Agent Embedding and LSTM Modelling.

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Surface Electromyography-based Gesture Recognition by Multi-view Deep Learning.

Deep Neural Networks with Broad Views for Parkinson's Disease Screening

Detection and Tracking of Multiple Mice Using Part Proposal Networks

Detecting and Tracking of Multiple Mice Using Part Proposal Networks

Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour

Interaction Behavior Recognition from Multiple Views

Multi-view and multi-scale behavior recognition algorithm based on attention mechanism

Structured Context Enhancement Network for Mouse Pose Estimation

Human Collective Intelligence Inspired Multi-View Representation Learning — Enabling View Communication by Simulating Human Communication Mechanism

Human Behavior Recognition from Multiview Videos

Markerless Mouse Tracking for Social Experiments

Mouse Behavior Recognition Based On Convolution Neural Networ

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice

Multi-view Ensemble Learning for Dementia Diagnosis from Neuroimaging: an Artificial Neural Network Approach

A novel mice interaction recognition method by using machine learning