Abstract:Advances in machine learning and contactless sensors have enabled the understanding complex human behaviors in a healthcare setting. In particular, several deep learning systems have been introduced to enable comprehensive analysis of neuro-developmental conditions such as Autism Spectrum Disorder (ASD). This condition affects children from their early developmental stages onwards, and diagnosis relies entirely on observing the child's behavior and detecting behavioral cues. However, the diagnosis process is time-consuming as it requires long-term behavior observation, and the scarce availability of specialists. We demonstrate the effect of a region-based computer vision system to help clinicians and parents analyze a child's behavior. For this purpose, we adopt and enhance a dataset for analyzing autism-related actions using videos of children captured in uncontrolled environments ( e.g. videos collected with consumer-grade cameras, in varied environments). The data is pre-processed by detecting the target child in the video to reduce the impact of background noise. Motivated by the effectiveness of temporal convolutional models, we propose both light-weight and conventional models capable of extracting action features from video frames and classifying autism-related behaviors by analyzing the relationships between frames in a video. By extensively evaluating feature extraction and learning strategies, we demonstrate that the highest performance is attained through the use of an Inflated 3D Convnet and Multi-Stage Temporal Convolutional Network. Our model achieved a Weighted F1-score of 0.83 for the classification of the three autism-related actions. We also propose a light-weight solution by employing the ESNet backbone with the same action recognition model, achieving a competitive 0.71 Weighted F1-score, and enabling potential deployment on embedded systems. Experimental results demonstrate the ability of our proposed models to recognize autism-related actions from videos captured in an uncontrolled environment, and thus can assist clinicians in analyzing ASD.

Spatial attentional bilinear 3d convolutional network for video-based autism spectrum disorder detection

A Comparative Study on Autism Spectrum Disorder Detection Via 3D Convolutional Neural Networks

Vision-based activity recognition in children with autism-related behaviors

Bilinear Perceptual Fusion Algorithm Based on Brain Functional and Structural Data for ASD Diagnosis and Regions of Interest Identification

Video-Based Autism Detection with Deep Learning

Autism spectrum disorder analysis by using a 3D-ResNet-based approach

Video-Based Early ASD Detection Via Temporal Pyramid Networks

A Two-stream End-to-End Deep Learning Network for Recognizing Atypical Visual Attention in Autism Spectrum Disorder

A multi-view convolutional neural network method combining attention mechanism for diagnosing autism spectrum disorder

Multi-View and Multimodal Graph Convolutional Neural Network for Autism Spectrum Disorder Diagnosis

Predicting Atypical Visual Saliency for Autism Spectrum Disorder Via Scale-Adaptive Inception Module and Discriminative Region Enhancement Loss

A Spatial-Temporal Dual-Mode Mixed Flow Network for Panoramic Video Salient Object Detection

Learning Attention-Enhanced Spatiotemporal Representation for Action Recognition

Multi-scale Spatial-temporal Interaction Network for Video Anomaly Detection

A Novel Dataset for Video-Based Autism Classification Leveraging Extra-Stimulatory Behavior

Interpretable Learning Approaches in Structural MRI: 3D-Resnet Fused Attention for Autism Spectrum Disorder Classification.

An Advanced Deep Learning Framework for Video-Based Diagnosis of ASD

Bimodal Feature Analysis with Deep Learning for Autism Spectrum Disorder Detection

Large-Scale Brain Functional Network Integration for Discrimination of Autism Using a 3-D Deep Learning Model

A novel approach for ASD recognition based on graph attention networks

Multi-scale Convolutional Network for fMRI-Based Diagnosis of Autism Spectrum Disorder