Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

3D-MuPPET: 3D Multi-Pigeon Pose Estimation and Tracking

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture

Benchmarking Monocular 3D Dog Pose Estimation Using In-The-Wild Motion Capture Data

Peering into the world of wild passerines with 3D-SOCS: synchronized video capture and posture estimation

Three-dimensional Surface Motion Capture of Multiple Freely Moving Pigs Using MAMMAL.

vmTracking: Virtual Markers Overcome Occlusion and Crowding in Multi-Animal Pose Tracking

A Real-time Multi-Subject Three Dimensional Pose Tracking System for Analyzing Social Behaviors of Non-human Primates

VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild

APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking

Context-Aware Outlier Rejection for Robust Multi-View 3D Tracking of Similar Small Birds in An Outdoor Aviary

MarmoPose: A Deep Learning-Based System for Real-time Multi-Marmoset 3D Pose Tracking

PMotion: An advanced markerless pose estimation approach based on novel deep learning framework used to reveal neurobehavior

MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detection

APTv2: Benchmarking Animal Pose Estimation and Tracking with a Large-scale Dataset and Beyond

[Three-dimensional positioning and trajectory tracking of pigeons in large indoor spaces]

Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild

Multi-camera Realtime 3D Tracking of Multiple Flying Animals

Multi-animal pose estimation and tracking with DeepLabCut

Multi-animal pose estimation, identification and tracking with DeepLabCut

Anti-drift pose tracker (ADPT): A transformer-based network for robust animal pose estimation cross-species