Abstract:Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to provide affordable solutions, though, it is important to minimize the number of sensors attached to the subject's body. Unfortunately, reconstructing the full-body pose from sparse data is a heavily under-determined problem. Some studies that use IMU sensors face challenges in reconstructing the pose due to positional drift and ambiguity of the poses. In recent years, some mainstream VR systems have released 6-degree-of-freedom (6-DoF) tracking devices providing positional and rotational information. Nevertheless, most solutions for reconstructing full-body poses rely on traditional inverse kinematics (IK) solutions, which often produce non-continuous and unnatural poses. In this article, we introduce SparsePoser, a novel deep learning-based solution for reconstructing a full-body pose from a reduced set of six tracking devices. Our system incorporates a convolutional-based autoencoder that synthesizes high-quality continuous human poses by learning the human motion manifold from motion capture data. Then, we employ a learned IK component, made of multiple lightweight feed-forward neural networks, to adjust the hands and feet toward the corresponding trackers. We extensively evaluate our method on publicly available motion capture datasets and with real-time live demos. We show that our method outperforms state-of-the-art techniques using IMU sensors or 6-DoF tracking devices, and can be used for users with different body dimensions and proportions.

NONINTRUSIVE VIRTUAL TRAINER PROTOTYPE FOR EXERCISE ROUTINES IN UNITY USING MOTION CAPTURE

A Motion-based User Interface for the Control of Virtual Humans Performing Sports.

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

VRInsole: an Unobtrusive and Immersive Mobility Training System for Stroke Rehabilitation

Building Virtual Entertainment Environment with Tiled Display Wall and Motion Tracking.

Implementation and Evaluation of Walk-in-Place Using a Low-Cost Motion-Capture Device for Virtual Reality Applications

Application of Virtual Reality and Artificial Intelligence Technology in Fitness Clubs

Trainer: a motion-based interactive game for balance rehabilitation training.

Motion Capture in Mixed-Reality Applications: A Deep Denoising Approach

Construction and Application of Virtual Reality-Based Sports Rehabilitation Training Program

Virtual Character Animation Based on Affordable Motion Capture and Reconfigurable Tangible Interfaces

An Approach to Physical Rehabilitation Using State-of-the-art Virtual Reality and Motion Tracking Technologies

Artificial intelligence based virtual gaming experience for sports training and simulation of human motion trajectory capture

Virtual Hand Training Platform Controlled Through Online Recognition of Motion Intention

Motion Matching for Character Animation and Virtual Reality Avatars in Unity

Integration of a Balanced Virtual Manikin in a Virtual Reality Platform aimed at Virtual Prototyping

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data

Virtual experiment system for electrician training based on Kinect and Unity3D

Unidirectional Human-Robot-Human Physical Interaction for Gait Training

Unity game engine: interactive software design using digital glove for virtual reality baseball pitch training