Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

Human Posture Tracking in a Multi-Camera Environment

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Real-time Follow-Up Head Tracking in Dynamic Complex Environments

Humans Tracking In The Complicated Background By Multi-Cue Integration

Efficient Multi-person Hierarchical 3D Pose Estimation for Autonomous Driving

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Human Body Recognition And Positioning With Multiple Cameras Based On "Vibration Signals" From Skin Surfaces

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Two-camera-based Human Motion Capture

Multi-person Multi-Camera Tracking for Live Stream Videos Based on Improved Motion Model and Matching Cascade

Human Posture Tracking and Classification through Stereo Vision and 3D Model Matching

Multiple Human Tracking Based on Multi-view Upper-Body Detection and Discriminative Learning

Human Motion Tracking by Multiple RGBD Cameras.

Multiple Human Association and Tracking from Egocentric and Complementary Top Views

A Top-down Approach to Articulated Human Pose Estimation and Tracking

Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

A Robust Approach for Person Localization in Multi-camera Environment

A Hierarchical Algorithm for Human Posture Recognition Based on Spatial and Frequency Domain Features

Multi-Branch High-Dimensional Guided Transformer-Based 3D Human Posture Estimation

Face Pose Tracking with 3D Model and Weighted Depth and Brightness Constraints