Abstract:Recent technological advancements have spurred the usage of machine learning based applications in sports science and healthcare. Using wearable sensors and video cameras to analyze and improve the performance of athletes, has become widely popular. Physiotherapists, sports coaches and athletes actively look to incorporate the latest technologies in order to further improve performance and avoid injuries. While wearable sensors are very popular, their use is hindered by constraints on battery power and sensor calibration, especially for use cases which require multiple sensors to be placed on the body. Hence, there is renewed interest in video-based data capture and analysis for sports science. In this paper, we present the application of classifying strength and conditioning exercises using video. We focus on the popular Military Press exercise, where the execution is captured with a video-camera using a mobile device, such as a mobile phone, and the goal is to classify the execution into different types. Since video recordings need a lot of storage and computation, this use case requires data reduction, while preserving the classification accuracy and enabling fast prediction. To this end, we propose an approach named BodyMTS to turn video into time series by employing body pose tracking, followed by training and prediction using multivariate time series classifiers. We analyze the accuracy and robustness of BodyMTS and show that it is robust to different types of noise caused by either video quality or pose estimation factors. We compare BodyMTS to state-of-the-art deep learning methods which classify human activity directly from videos and show that BodyMTS achieves similar accuracy, but with reduced running time and model engineering effort. Finally, we discuss some of the practical aspects of employing BodyMTS in this application in terms of accuracy and robustness under reduced data quality and size. We show that BodyMTS achieves an average accuracy of 87%, which is significantly higher than the accuracy of human domain experts.

ExerSense: Real-Tme Physical Exercise Segmentation, Classification, and Counting Algorithm Using an IMU Sensor

Seven Things to Know about Exercise Classification with Inertial Sensing Wearables

An Examination of Wearable Sensors and Video Data Capture for Human Exercise Classification

Intelligent Repetition Counting for Unseen Exercises: A Few-Shot Learning Approach with Sensor Signals

Rehabilitation Exercise Repetition Segmentation and Counting using Skeletal Body Joints

Recognition and Scoring Physical Exercises via Temporal and Relative Analysis of Skeleton Nodes Extracted from the Kinect Sensor

Real-Time Fitness Exercise Classification and Counting from Video Frames

Deeper Exercise Monitoring for Smart Gym using Fused RFID and CV Data

Simultaneous exercise recognition and evaluation in prescribed routines: Approach to virtual coaches

Otago Exercises Monitoring for Older Adults by a Single IMU and Hierarchical Machine Learning Models

Automatic Classification of Knee Rehabilitation Exercises Using a Single Inertial Sensor: a Case Study

Recognizing Exercises and Counting Repetitions in Real Time

Development of Wearable Devices for Collecting Digital Rehabilitation/Fitness Data from Lower Limbs

Realtime Indoor Workout Analysis Using Machine Learning & Computer Vision

DS-MS-TCN: Otago Exercises Recognition with a Dual-Scale Multi-Stage Temporal Convolutional Network

Fast and robust video-based exercise classification via body pose tracking and scalable multivariate time series classifiers

A 'one-size-fits-most' walking recognition method for smartphones, smartwatches, and wearable accelerometers

Multimodal sensor fusion models for real-time exercise repetition counting with IMU sensors and respiration data

Towards Automating Personal Exercise Assessment and Guidance with Affordable Mobile Technology

Pūioio: On-device Real-Time Smartphone-Based Automated Exercise Repetition Counting System