Sao Mai Nguyen,Maxime Devanne,Olivier Remy-Neris,Mathieu Lempereur,André Thepaut
Abstract:While automatic monitoring and coaching of exercises are showing encouraging results in non-medical applications, they still have limitations such as errors and limited use contexts. To allow the development and assessment of physical rehabilitation by an intelligent tutoring system, we identify in this article four challenges to address and propose a medical dataset of clinical patients carrying out low back-pain rehabilitation exercises. The dataset includes 3D Kinect skeleton positions and orientations, RGB videos, 2D skeleton data, and medical annotations to assess the correctness, and error classification and localisation of body part and timespan. Along this dataset, we perform a complete research path, from data collection to processing, and finally a small benchmark. We evaluated on the dataset two baseline movement recognition algorithms, pertaining to two different approaches: the probabilistic approach with a Gaussian Mixture Model (GMM), and the deep learning approach with a Long-Short Term Memory (LSTM).
This dataset is valuable because it includes rehabilitation relevant motions in a clinical setting with patients in their rehabilitation program, using a cost-effective, portable, and convenient sensor, and because it shows the potential for improvement on these challenges.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the challenges in automatic monitoring and guiding physical rehabilitation exercises, especially in low - back pain rehabilitation training. Specifically, the author aims to address four main challenges by creating a medical dataset:
1. **Rehabilitation motion assessment**: Determine whether the rehabilitation exercises performed by patients are correct.
2. **Rehabilitation error recognition**: Classify the observed errors into known error types for the purpose of explanation and providing feedback.
3. **Spatial localization of the error**: Identify the body part that causes the error.
4. **Temporal localization of the error**: Detect the specific time period when the error occurs.
These problems have not been fully addressed in existing datasets, especially in the medical context. To meet these challenges, the author proposes a dataset named Keraal, which includes low - back pain rehabilitation exercise data from clinical patients, as well as detailed annotation information, such as 3D Kinect skeleton positions and orientations, RGB videos, 2D skeleton data, and medical annotations.
### Characteristics of the dataset
- **Multimodal data**: It includes 3D Kinect skeleton positions and orientations, RGB videos, 2D skeleton data, and Vicon MoCap data.
- **Detailed annotations**: Each movement is annotated by doctors and physiotherapists, including whether the movement is correct, the error type, the body part where the error occurs, and the time range.
- **Clinical background**: The data is sourced from patients actually participating in rehabilitation treatment, ensuring the authenticity and representativeness of the data.
### Benchmark tests
To evaluate the effectiveness of this dataset, the author uses two baseline algorithms for comparative experiments:
1. **Gaussian Mixture Model (GMM)**: A probabilistic method based on Riemannian manifolds.
2. **Long Short - Term Memory (LSTM)**: A method based on deep learning.
The experimental results show that LSTM performs better than GMM in most cases, but there is still room for improvement in certain specific movements (such as lateral flexion). This indicates that although the existing methods have certain effects, they still have limitations and further research is needed to improve accuracy.
### Conclusions and future work
This dataset provides a valuable resource for developing more intelligent rehabilitation tutoring systems and helps promote research in fields such as computer vision, machine learning, robotics, virtual agents, physical medicine, and biomechanics. Future work will focus on designing more accurate methods to assess rehabilitation movements and distinguish between subtle errors and correct movements, while expanding the types of rehabilitation exercises and the number of samples in the dataset.
By solving these problems, this research is expected to promote the development of telemedicine, enabling more patients to obtain high - quality rehabilitation guidance and support.