Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

PCA-based 3D Pose Modeling for Beating Heart Tracking

3D head tracking using SIFT-based registration and scale invariant view-based appearance model

Simultaneous segmentation and motion recovery in 3d cardiac image analysis

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

A Simultaneous Framework for Recovering Three Dimensional Shape and Nonrigid Motion from Cardiac Image Sequences

Spatio-temporal Active Region Model for Simultaneous Segmentation and Motion Estimation of the Whole Heart

Joint Segmentation and Motion Estimation of Left Ventricle with an Anisotropic Biomechanical Model

Joint Analysis of Heart Geometry and Kinematics with Spatiotemporal Active Region Model

Automated Heartbeat Classification Using 3-D Inputs Based on Convolutional Neural Network with Multi-Fields of View.

Spatiotemporal Strategies for Joint Segmentation and Motion Tracking from Cardiac Image Sequences

Cardiac Motion Analysis Using Nonlinear Biomechanical Constraints

Tracking Anything in Heart All at Once

Motion Prediction for Beating Heart Surgery with GRU

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

Continuous 3D Myocardial Motion Tracking via Echocardiography

3D Articulated Hand Tracking Based on Behavioral Model

Anatomy-Aware Cardiac Motion Estimation

Robotic Motion Compensation for Beating Heart Intracardiac Surgery

Automatic cardiac motion tracking using both untagged and 3d tagged MR images

Visual recognition of cardiac pathology based on 3D parametric model reconstruction

A Comprehensive Cardiac Motion Estimation Framework Using Both Untagged and 3-D Tagged MR Images Based on Nonrigid Registration