Abstract:Recent technological advancements have spurred the usage of machine learning based applications in sports science and healthcare. Using wearable sensors and video cameras to analyze and improve the performance of athletes, has become widely popular. Physiotherapists, sports coaches and athletes actively look to incorporate the latest technologies in order to further improve performance and avoid injuries. While wearable sensors are very popular, their use is hindered by constraints on battery power and sensor calibration, especially for use cases which require multiple sensors to be placed on the body. Hence, there is renewed interest in video-based data capture and analysis for sports science. In this paper, we present the application of classifying strength and conditioning exercises using video. We focus on the popular Military Press exercise, where the execution is captured with a video-camera using a mobile device, such as a mobile phone, and the goal is to classify the execution into different types. Since video recordings need a lot of storage and computation, this use case requires data reduction, while preserving the classification accuracy and enabling fast prediction. To this end, we propose an approach named BodyMTS to turn video into time series by employing body pose tracking, followed by training and prediction using multivariate time series classifiers. We analyze the accuracy and robustness of BodyMTS and show that it is robust to different types of noise caused by either video quality or pose estimation factors. We compare BodyMTS to state-of-the-art deep learning methods which classify human activity directly from videos and show that BodyMTS achieves similar accuracy, but with reduced running time and model engineering effort. Finally, we discuss some of the practical aspects of employing BodyMTS in this application in terms of accuracy and robustness under reduced data quality and size. We show that BodyMTS achieves an average accuracy of 87%, which is significantly higher than the accuracy of human domain experts.

Real-Time Fitness Exercise Classification and Counting from Video Frames

Fast and robust video-based exercise classification via body pose tracking and scalable multivariate time series classifiers

An Examination of Wearable Sensors and Video Data Capture for Human Exercise Classification

Realtime Indoor Workout Analysis Using Machine Learning & Computer Vision

ExerTrack—Towards Smart Surfaces to Track Exercises

Recognizing Exercises and Counting Repetitions in Real Time

ExerSense: Real-Tme Physical Exercise Segmentation, Classification, and Counting Algorithm Using an IMU Sensor

Workout Classification Using a Convolutional Neural Network in Ensemble Learning

Real-Time Sensor-Based Human Activity Recognition for eFitness and eHealth Platforms

Multimodal sensor fusion models for real-time exercise repetition counting with IMU sensors and respiration data

Rehabilitation Exercise Repetition Segmentation and Counting using Skeletal Body Joints

Fitness Done Right: a Real-time Intelligent Personal Trainer for Exercise Correction

Intelligent Repetition Counting for Unseen Exercises: A Few-Shot Learning Approach with Sensor Signals

Video-based Exercise Classification and Activated Muscle Group Prediction with Hybrid X3D-SlowFast Network

Pūioio: On-device Real-Time Smartphone-Based Automated Exercise Repetition Counting System

Exploration of deep learning architectures for real-time yoga pose recognition

IoT-Based Wearable Sensors and Bidirectional LSTM Network for Action Recognition of Aerobics Athletes

Wearable Sensor Data for Classification and Analysis of Functional Fitness Exercises Using Unsupervised Deep Learning Methodologies

Deep Learning for Fitness

Physiotherapy Exercise Classification with Single-Camera Pose Detection and Machine Learning

DS-MS-TCN: Otago Exercises Recognition with a Dual-Scale Multi-Stage Temporal Convolutional Network