Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

Articulated point pattern matching in optical motion capture systems

Articulated motion reconstruction from feature points

Articulated pose identification with sparse point features

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Reconstruction of segmentally articulated structure in freeform movement with low density feature points

Motion Analysis of Articulated Objects from Monocular Images.

Point Pattern Matching for Articulated or Multiple Objects

Automatic Marker Registration and Prediction Algorithm for Optical Motion Capture System

Approach of Scattered Data Processing for Optical Motion Capture

A part-based online modeling algorithm and its applications on robust visual tracking

Articulated Object Registration Using Simulated Physical Force/Moment For 3d Human Motion Tracking

3D Point Pattern Matching Based on Spatial Geometric Flexibility.

Graph Matching for Marker Labeling and Missing Marker Reconstruction with Bone Constraint by LSTM in Optical Motion Capture

Automatic Gait Motion Capture with Missing-Marker Fillings

Sequential Articulated Motion Reconstruction from a Monocular Image Sequence

3D Articulated Structure and Motion Analysis from Monocular Images

A Rigid Structure Matching-Based Noise Data Processing Approach for Human Motion Capture

Robust Human Pose Recognition Using Unlabelled Markers

VoCAPTER: Voting-based Pose Tracking for Category-level Articulated Object Via Inter-frame Priors

Towards Unstructured Unlabeled Optical Mocap: A Video Helps!

Autonomous Real-Time Model Building for Optical Motion Capture