Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations

David Schneider,Simon Reiß,Marco Kugler,Alexander Jaus,Kunyu Peng,Susanne Sutschet,M. Saquib Sarfraz,Sven Matthiesen,Rainer Stiefelhagen

2024-11-01

Abstract:Exploring the intricate dynamics between muscular and skeletal structures is pivotal for understanding human motion. This domain presents substantial challenges, primarily attributed to the intensive resources required for acquiring ground truth muscle activation data, resulting in a scarcity of datasets. In this work, we address this issue by establishing Muscles in Time (MinT), a large-scale synthetic muscle activation dataset. For the creation of MinT, we enriched existing motion capture datasets by incorporating muscle activation simulations derived from biomechanical human body models using the OpenSim platform, a common approach in biomechanics and human motion research. Starting from simple pose sequences, our pipeline enables us to extract detailed information about the timing of muscle activations within the human musculoskeletal system. Muscles in Time contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We demonstrate the utility of this dataset by presenting results on neural network-based muscle activation estimation from human pose sequences with two different sequence-to-sequence architectures. Data and code are provided under <a class="link-external link-https" href="https://simplexsigil.github.io/mint" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the problem of understanding the complex dynamic relationships between muscle and skeletal structures during human movement. Specifically, the authors focus on how to enhance the understanding of human movement through simulating muscle activation. The main challenge lies in the fact that the resources required to obtain real - muscle activation data are very intensive, resulting in a scarcity of relevant datasets.

To solve this problem, the authors propose the **Muscles in Time (MinT)** dataset, which is a large - scale synthetic muscle activation dataset. The creation process of MinT includes the following key steps:

1. **Enrich existing motion - capture datasets**: Use the OpenSim platform to extract muscle activation simulations from biomechanical human models, thereby adding detailed muscle activation information to existing motion - capture datasets.
2. **Generate detailed time - series data**: Starting from simple pose sequences, extract the time information of muscle activation in the human musculoskeletal system.
3. **Provide a rich amount of data**: MinT contains more than 9 hours of simulation data, covering 227 subjects and 402 simulated muscle fibers.

Through these methods, the authors hope to弥补 the deficiencies of existing EMG and sEMG datasets, which are usually small in scale, limited in scope and unable to represent a variety of human movements. The goal of the MinT dataset is to provide researchers with a large - scale, highly - detailed synthetic dataset to support more in - depth research on the relationship between muscle activation patterns and human movement.

In addition, the authors also show how to use the MinT dataset to train neural networks so that they can estimate muscle activation from human pose sequences, and evaluate the performance of different sequence - to - sequence architectures. This provides a new way to further explore the integration of muscle activation data into human movement understanding models.

In summary, the main contributions of this paper are:
- Creating a large - scale synthetic muscle activation dataset (MinT).
- Providing detailed muscle activation time - series data, which helps to more comprehensively understand human movement.
- Demonstrating the potential of neural networks in estimating muscle activation from pose sequences.