Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

An Extraction Algorithm for Motion Parameters of A Laboratory Mouse by Model Matching and Particle Filtering

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

A Video Tracking System for Limb Motion Measurement in Small Animals

Detecting and Tracking of Multiple Mice Using Part Proposal Networks

Motion States Extraction with Optical Flow for Rat-Robot Automatic Navigation

Detection and Tracking of Multiple Mice Using Part Proposal Networks

Markerless Mouse Tracking for Social Experiments

Human Motion Tracking Based on Statistical Model and Genetic Particle Filter

Multiple Dense Particle Tracking In Fluorescence Microscopy Images Based On Multidimensional Assignment

STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation

Optical Mouse: 3D Mouse Pose From Single-View Video

Whisker Analysis Framework for Unrestricted Mice with Neural Networks

Feature Based Framework to Detect Diseases, Tumor, and Bleeding in Wireless Capsule Endoscopy

2D Articulated Pose Tracking Using Particle Filter with Partitioned Sampling and Model Constraints

Micro-object motion tracking based on the probability hypothesis density particle tracker

Murine Motion Behavior Recognition Based on DeepLabCut and Convolutional Long Short-Term Memory Network

Apparatus and methods for mouse behavior recognition on foot contact features

Motion Artifact Correction of Multi-Photon Imaging of Awake Mice Models Using Speed Embedded HMM

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

High-Precision, Three-Dimensional Tracking of Mouse Whisker Movements with Optical Motion Capture Technology

Behavior Recognition in Mouse Videos Using Contextual Features Encoded by Spatial-temporal Stacked Fisher Vectors.