Abstract:Marker-based motion capture (mocap) is a conventional method used in biomechanics research to precisely analyze human movement. However, the time-consuming marker placement process and extensive post-processing limit its wider adoption. Therefore, markerless mocap systems that use deep learning to estimate 2D keypoint from images have emerged as a promising alternative, but annotation errors in training datasets used by deep learning models can affect estimation accuracy. To improve the precision of 2D keypoint annotation, we present a method that uses anatomical landmarks based on marker-based mocap. Specifically, we use multiple RGB cameras synchronized and calibrated with a marker-based mocap system to create a high-quality dataset (RRIS40) of images annotated with surface anatomical landmarks. A deep neural network is then trained to estimate these 2D anatomical landmarks and a ray-distance-based triangulation is used to calculate the 3D marker positions. We conducted extensive evaluations on our RRIS40 test set, which consists of 10 subjects performing various movements. Compared against a marker-based system, our method achieves a mean Euclidean error of 13.23 mm in 3D marker position, which is comparable to the precision of marker placement itself. By learning directly to predict anatomical keypoints from images, our method outperforms OpenCap's augmentation of 3D anatomical landmarks from triangulated wild keypoints. This highlights the potential of facilitating wider integration of markerless mocap into biomechanics research. The RRIS40 test set is made publicly available for research purposes at koonyook.github.io/rris40.

Implement of the transformation for markerless facial motion capture in 3D animation

Animating Scanned Human Models.

A Piece-Wise Learning Approach To 3d Facial Animation

Animation of 3D Human Model Using Markerless Motion Capture Applied To Sports

Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras

Video Tracked Facial Expression Animation

Motion Capture Technique with Enhancement Filters for Humanoid Model Movement Animation

Anatomical-Marker-Driven 3D Markerless Human Motion Capture

Video-driven state-aware facial animation

Quick Customization of Particular Human Face and Muscle-Driven Expression Animation

A Method of Facial Animation Retargeting Based on Motion Capture

Multi-view Human Motion Capture with an Improved Deformation Skin Model

MarkerNet: A divide‐and‐conquer solution to motion capture solving from raw markers

Hierarchical facial expression animation by motion capture data

Facial animation by optimized blendshapes from motion capture data

A Review of Deep Learning Techniques for Markerless Human Motion on Synthetic Datasets

3D Shape Regression for Real-Time Facial Animation

Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with an RGB-D camera

Semi-supervised video-driven facial animation transfer for production

Transferring of Speech Movements from Video to 3D Face Space

Look Ma, no markers: holistic performance capture without the hassle