Abstract:Motion parameters measurement is essential for understanding animal behavior, exploring the laws of object motion, and studying control methods. Nowadays, advanced computer vision based on machine learning technology supports markerless object tracking in 2D videos. However, due to the fact that all objects move in three-dimensional space, this paper introduces a method of measuring motion parameters using 3D pose estimation. First, an enhanced iterative bundle adjustment algorithm is proposed for multi-camera calibration in a multi-camera vision system by adding two control parameters, which dramatically reduces the reprojection error of multi-camera calibration and lays the foundation for high-precision triangulation. Then, a new spatiotemporal loss function is proposed, which considers the relationship between key points that do not constitute limbs, thereby improving triangulation accuracy. The new multi-camera calibration algorithm is evaluated on ChArUco and 3D pose estimation for metronome, planet pendulum, human hand, Koi, and cheetah. The experimental results show that: (1) the two hyper-parameters in the enhanced iterative bundle adjustment algorithm effectively suppress the influence of noise and play a good role in reducing the reprojection error of multi-camera calibration; (2) the spatiotemporal loss function has a strong constraining ability, the time loss can stabilize high frame rate video triangulation to maintain accuracy, while the space loss can improve the accuracy of triangulation for more complex structures; (3) multi-view data fusion is also conducive to improving the accuracy of triangulation. Moreover, the method was successfully applied to some actual measurement scenes: (1) the accurate measurement of the frequency of a metronome; and (2) the success measurement of the movement of a Koi, which conforms to the basic model of fish swimming. Some dynamic measurement results are displayed at https://github.com/wux024/AdamPose .

Research on Key Point Estimation of Interactive Pen Based on Deep Learning

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Key Point Detection in 3D Reconstruction Based on Human-Computer Interaction

PKLNet: Keypoint Localization Neural Network for Touchless Palmprint Recognition Based on Edge-Aware Regression

Accurate Robot Arm Attitude Estimation Based on Multi-View Images and Super-Resolution Keypoint Detection Networks

A Mask RCNN based Automatic Reading Method for Pointer Meter

A Pen-Based Interactive Projection System Based on the Infrared Scanning Projector.

Indirect real-time measurement of the penpoint in an interactive projector-camera system

Neural Interactive Keypoint Detection

Three-Point Light Pen Measurement Method Based On Light Field Imaging

Depth-Hand: 3D Hand Keypoint Detection with Dense Depth Estimation

Two Heads Are Better than One: Image-Point Cloud Network for Depth-Based 3D Hand Pose Estimation

Kpdet: Keypoint-Based 3d Object Detection with Parametric Radius Learning

VIEWPOINT ESTIMATION IN IMAGES BY A KEY-POINT BASED DEEP NEURAL NETWORK

Brain Inspired Keypoint Matching for 3D Scene Reconstruction

3D Keypoint Detection Based on Deep Neural Network with Sparse Autoencoder

Position And Orientation Estimation Of Palm Based On Mark Points

Key Points Detection Algorithm of Object Based on Full Convolution Network

A Framework to Estimate the Key Point Within an Object Based on a Deep Learning Object Detection

A Method for Detecting Key Points of Transferring Barrel Valve by Integrating Keypoint R-CNN and MobileNetV3