Abstract:Advances in machine learning and contactless sensors have enabled the understanding complex human behaviors in a healthcare setting. In particular, several deep learning systems have been introduced to enable comprehensive analysis of neuro-developmental conditions such as Autism Spectrum Disorder (ASD). This condition affects children from their early developmental stages onwards, and diagnosis relies entirely on observing the child's behavior and detecting behavioral cues. However, the diagnosis process is time-consuming as it requires long-term behavior observation, and the scarce availability of specialists. We demonstrate the effect of a region-based computer vision system to help clinicians and parents analyze a child's behavior. For this purpose, we adopt and enhance a dataset for analyzing autism-related actions using videos of children captured in uncontrolled environments ( e.g. videos collected with consumer-grade cameras, in varied environments). The data is pre-processed by detecting the target child in the video to reduce the impact of background noise. Motivated by the effectiveness of temporal convolutional models, we propose both light-weight and conventional models capable of extracting action features from video frames and classifying autism-related behaviors by analyzing the relationships between frames in a video. By extensively evaluating feature extraction and learning strategies, we demonstrate that the highest performance is attained through the use of an Inflated 3D Convnet and Multi-Stage Temporal Convolutional Network. Our model achieved a Weighted F1-score of 0.83 for the classification of the three autism-related actions. We also propose a light-weight solution by employing the ESNet backbone with the same action recognition model, achieving a competitive 0.71 Weighted F1-score, and enabling potential deployment on embedded systems. Experimental results demonstrate the ability of our proposed models to recognize autism-related actions from videos captured in an uncontrolled environment, and thus can assist clinicians in analyzing ASD.

Vision-Based Action Detection for RTI Protocol of ASD Early Screening

Early Screening of Autism in Toddlers Via Response-To-Instructions Protocol

Vision-based activity recognition in children with autism-related behaviors

ASDPred: an End-to-End Autism Screening Framework Using Few-Shot Learning

Effective And Efficient Visual Stimuli Design For Quantitative Autism Screening: An Exploratory Study

Video-Based Early ASD Detection Via Temporal Pyramid Networks

Vision-Based Pointing Estimation and Evaluation in Toddlers for Autism Screening

Automatic Classification of ASD Children Using Appearance-Based Features from Videos

Identification of ASD Children Based on Video Data.

Early Diagnosis of ASD Based on Facial Expression Recognition and Head Pose Estimation

Gaze-Wasserstein: a Quantitative Screening Approach to Autism Spectrum Disorders

Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

Enhancing Autism Spectrum Disorder Early Detection with the Parent-Child Dyads Block-Play Protocol and an Attention-enhanced GCN-xLSTM Hybrid Deep Learning Framework

Computer vision tools for the non-invasive assessment of autism-related behavioral markers

Uncertainty Inspired Autism Spectrum Disorder Screening

An Auxiliary Screening System for Autism Spectrum Disorder Based on Emotion and Attention Analysis

SP-ASDNet: CNN-LSTM Based ASD Classification Model Using Observer ScanPaths

ViTASD: Robust Vision Transformer Baselines for Autism Spectrum Disorder Facial Diagnosis

Dynamic Viewing Pattern Analysis: Towards Large-Scale Screening of Children with ASD in Remote Areas

Visual Question Answering Driven Eye Tracking Paradigm for Identifying Children with Autism Spectrum Disorder

Classifying ASD Children with LSTM Based on Raw Videos