Abstract:Advances in machine learning and contactless sensors have enabled the understanding complex human behaviors in a healthcare setting. In particular, several deep learning systems have been introduced to enable comprehensive analysis of neuro-developmental conditions such as Autism Spectrum Disorder (ASD). This condition affects children from their early developmental stages onwards, and diagnosis relies entirely on observing the child's behavior and detecting behavioral cues. However, the diagnosis process is time-consuming as it requires long-term behavior observation, and the scarce availability of specialists. We demonstrate the effect of a region-based computer vision system to help clinicians and parents analyze a child's behavior. For this purpose, we adopt and enhance a dataset for analyzing autism-related actions using videos of children captured in uncontrolled environments ( e.g. videos collected with consumer-grade cameras, in varied environments). The data is pre-processed by detecting the target child in the video to reduce the impact of background noise. Motivated by the effectiveness of temporal convolutional models, we propose both light-weight and conventional models capable of extracting action features from video frames and classifying autism-related behaviors by analyzing the relationships between frames in a video. By extensively evaluating feature extraction and learning strategies, we demonstrate that the highest performance is attained through the use of an Inflated 3D Convnet and Multi-Stage Temporal Convolutional Network. Our model achieved a Weighted F1-score of 0.83 for the classification of the three autism-related actions. We also propose a light-weight solution by employing the ESNet backbone with the same action recognition model, achieving a competitive 0.71 Weighted F1-score, and enabling potential deployment on embedded systems. Experimental results demonstrate the ability of our proposed models to recognize autism-related actions from videos captured in an uncontrolled environment, and thus can assist clinicians in analyzing ASD.

Baby Physical Safety Monitoring in Smart Home Using Action Recognition System

Challenges in Video-Based Infant Action Recognition: A Critical Examination of the State of the Art

E-BabyNet: Enhanced Action Recognition of Infant Reaching in Unconstrained Environments

Human Action Recognition From Digital Videos Based on Deep Learning.

A Skeleton-based Action Recognition System for Medical Condition Detection

Human Action Recognition Using Deep Learning Methods.

Vision-based activity recognition in children with autism-related behaviors

Action Recognition in Videos through a Transfer-Learning-Based Technique

Visual Context Based Infant Activity Analysis

Adapting Action Recognition Neural Networks for Automated Infantile Spasm Detection

A Review and Comparative Study of Explainable Deep Learning Models Applied on Action Recognition in Real Time

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

Deep Learning-Based Human Action Recognition in Videos

PoseAction: Action Recognition for Patients in the Ward using Deep Learning Approaches

Skeleton‐Guided Action Recognition with Multistream 3D Convolutional Neural Network for Elderly‐Care Robot

Deep Learning-Based Real-Time Multiple-Person Action Recognition System

Deep Learning-based human activity recognition using RGB images in Human-robot collaboration

Learning and Recognizing Human Action from Skeleton Movement with Deep Residual Neural Networks

A Novel Prediction Method of Transfer-Assisted Action Oriented to Individual Differences for the Excretion Care Robot

Computational Baby Learning

Activity Recognition From Newborn Resuscitation Videos