Abstract:We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).

Human Upper Limb Motion Analysis for Post-Stroke Impairment Assessment Using Video Analytics.

Upper limb movement analysis via marker tracking with a single-camera system

A Depth Camera Motion Analysis Framework for Tele-Rehabilitation: Motion Capture and Person-Centric Kinematics Analysis

Automated Movement Assessment in Stroke Rehabilitation

Kinematics Analysis Multimedia System for Rehabilitation.

Validation of Consumer-grade Digital Camera-based Human Activity Evaluation for Upper Limb Exercises and Development of a Therapist-guided, Automated Telerehabilitation Framework and Platform for Stroke Rehabilitation

Abnormal Motion Capture in Acute Stroke (BIONICS): A Low-Cost Tele-Evaluation Tool for Automated Assessment of Upper Extremity Function in Stroke Patients.

Novel Digital Assessment System for Upper-Limb Movement in Stroke Patients Using Markless-Sensing Technology and Deep Learning Algorithms

Gait Analysis Using a Single Depth Camera

Markerless Motion Capture System for Stroke Gait Analysis

Automation Enhancement and Accuracy Investigation of a Portable Single‐camera Gait Analysis System

The Use of Head-Mounted Display Systems for Upper Limb Kinematic Analysis in Post-Stroke Patients: A Perspective Review on Benefits, Challenges and Other Solutions

Development of a Novel Home Based Multi-Scene Upper Limb Rehabilitation Training and Evaluation System for Post-Stroke Patients.

Opportunities for Improving Motor Assessment and Rehabilitation After Stroke by Leveraging Video-Based Pose Estimation

A Fine-Grained Analysis of Post-Stroke Motor Function Using Wearable Motion Sensors

Computer-Assisted Hand Rehabilitation Assessment Using an Optical Motion Capture System

Video-Based Pose Estimation for Gait Analysis in Stroke Survivors during Clinical Assessments: A Proof-of-Concept Study

Quantitative Upper Limb Impairment Assessment for Stroke Rehabilitation: A Review

Vision-based Automatic Detection of Compensatory Postures of after-Stroke Patients During Upper-extremity Robot-assisted Rehabilitation: A Pilot Study in Reaching Movement

Automatic Assessment of Upper Extremity Function and Mobile Application for Self-Administered Stroke Rehabilitation

An Automated System for Motor Function Assessment in Stroke Patients Using Motion Sensing Technology: A Pilot Study