Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

Tracking End-Effectors for Marker-Less 3D Human Motion Estimation in Multi-view Image Sequences.

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Three-dimensional head pose tracking using registration and multi-scale appearance model

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Adaptive Particle Filter With Body Part Segmentation For Full Body Tracking

Visual end-effector tracking using a 3D model-aided particle filter for humanoid robot platforms

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

Towards Accurate Markerless Human Shape and Pose Estimation over Time

Human Motion Estimation from a Reduced Marker Set.

Human motion tracking based on shape analysis

High-Accuracy Real-Time Whole-Body Human Motion Tracking Based on Constrained Nonlinear Kalman Filtering

Reconstruction of 3D Human Motion Using Multiple View L1 Trackers

Markerless 3D human pose tracking through multiple cameras and AI: Enabling high accuracy, robustness, and real-time performance

3D Model Based Head Pose Tracking by Using Weighted Depth and Brightness Constraints

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

2D-3D Pose Tracking with Multi-View Constraints

Multi-view Human Motion Capture with an Improved Deformation Skin Model

A Markerless Human-Manipulators Interface Using Multi-Sensors.

Marker-Less Human Body Tracking Using Locally Affine Invariant Contour Matching

New multi-view human motion capture framework

Motion-Driven Tracking via End-to-End Coarse-to-Fine Verifying