Explainable Artificial Intelligence for Quantifying Interfering and High-Risk Behaviors in Autism Spectrum Disorder in a Real-World Classroom Environment Using Privacy-Preserving Video Analysis

Barun Das,Conor Anderson,Tania Villavicencio,Johanna Lantz,Jenny Foster,Theresa Hamlin,Ali Bahrami Rad,Gari D. Clifford,Hyeokhyen Kwon

2024-07-31

Abstract:Rapid identification and accurate documentation of interfering and high-risk behaviors in ASD, such as aggression, self-injury, disruption, and restricted repetitive behaviors, are important in daily classroom environments for tracking intervention effectiveness and allocating appropriate resources to manage care needs. However, having a staff dedicated solely to observing is costly and uncommon in most educational settings. Recently, multiple research studies have explored developing automated, continuous, and objective tools using machine learning models to quantify behaviors in ASD. However, the majority of the work was conducted under a controlled environment and has not been validated for real-world conditions. In this work, we demonstrate that the latest advances in video-based group activity recognition techniques can quantify behaviors in ASD in real-world activities in classroom environments while preserving privacy. Our explainable model could detect the episode of problem behaviors with a 77% F1-score and capture distinctive behavior features in different types of behaviors in ASD. To the best of our knowledge, this is the first work that shows the promise of objectively quantifying behaviors in ASD in a real-world environment, which is an important step toward the development of a practical tool that can ease the burden of data collection for classroom staff.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem of rapid identification and accurate recording of disruptive and high - risk behaviors (such as aggression, self - harm, destructiveness, and restricted repetitive behaviors) exhibited by patients with Autism Spectrum Disorder (ASD) in real - world classroom environments. Specifically, the authors focus on the following key issues:

1. **Behavior assessment and intervention effect tracking**:
- In the daily classroom environment, rapid and accurate identification and recording of these behaviors are crucial for tracking the effects of intervention measures and rationally allocating resources to manage care needs.

2. **Limitations of human observation**:
- The current standard practice is to determine the triggers, antecedents, and consequences of behaviors through Functional Behavior Assessment (FBA). However, relying on full - time personnel for observation is both expensive and uncommon, especially in most educational environments.
- Human observation also has problems of subjectivity and inconsistency, and the inter - rater reliability of certain autistic behaviors is low.

3. **Need for automated tools**:
- Research shows that machine - learning models can be used to automatically quantify behaviors in ASD, but most studies are conducted in controlled environments and have not been verified under real - world conditions.
- Therefore, there is a need to develop a tool that can objectively and automatically quantify ASD - related behaviors in real - world classroom environments.

4. **Privacy protection**:
- When using video analysis technology, ensuring that personal privacy is not violated is an important consideration.

### Main contributions of the paper

- **First application of advanced video analysis technology in real - world classroom environments**: The authors show how to use the latest video group activity recognition technology to quantify the disruptive and high - risk behaviors of ASD patients while protecting privacy.
- **Application of Explainable Artificial Intelligence (XAI)**: A model that can be explained is proposed, which can detect the occurrence of problem behaviors and capture the unique characteristics of different types of behaviors.
- **Efficient behavior detection**: This model achieves a 77% F1 score in detecting problem behaviors, showing good performance.

Through these methods, the paper provides an important step in developing practical tools that can help reduce the data - collection burden on classroom staff and improve the accuracy and efficiency of behavior assessment.