Abstract:IS (Interpersonal Synchrony), where the follower (participant) tries to behave the same action along with the raiser (human or metronome), is an essential social interaction skill. The evaluation of interpersonal synchronization is valuable for early autism screening. However, the research on IS evaluation is limited, and the current approaches usually evaluate the IS task with "motion energy" that is calculated by imprecise corner detection of the participant, which is not robust in an uncontrollable clinical environment. Moreover, these approaches need to manually mark the start and the end anchor of the specified action segment, which is labor-intensive. In this paper, we construct a realtime action segmentation model to automatically recognize the human-wise action class frame by frame. A simple yet efficient backbone is utilized to classify action class straightly instead of extracting the motion features (e.g. optical flow) with high computational complexity. Specifically, given an action video, a sliding window stacks frames in a fixed window size to feed a Resnet-like action classification branch (ACB) to classify the current action label. To further improve the accuracy of action boundary and eliminate the over-segmentation noises, we incorporate a boundary prediction branch (BPB), cooperating with majority-voting strategy, to refine the action classification generated by ACB. Then we can calculate the IS overlap easily by comparing two action timelines belonging to raiser and follower. To evaluate the proposed model, we collect 200K annotated images belonging to 40 subjects who perform 2 tasks (nod and clap) in 2 conditions (interpersonal and human-metronome). The experiment results demonstrate that our model achieves 87.1% accuracy at 200 FPS and can locate the start and end of action precisely in realtime.

Synchrony Expression Analysis of Human-Human Interaction Using Wearable Sensors.

Human-human Interactional Synchrony Analysis Based on Body Sensor Networks

Activity Recognition and Coordination Analysis of Two-Body Interactions Using Wearable Sensors

Analyzing Temporal Dynamics of Dyadic Synchrony in Affective Interactions.

A Two-Step Shapelets Based Framework for Interactional Activities Recognition

In Sync: Exploring Synchronization to Increase Trust Between Humans and Non-humanoid Robots

Collaborative Learning Quality Classification Through Physiological Synchrony Recorded by Wearable Biosensors

Wearable and Ambient Sensor Fusion for the Characterisation of Human Motion

A Long-Term Synchronized System for Healthcare

Hybrid Harmony: A Multi-Person Neurofeedback Application for Interpersonal Synchrony

A Wearable Electrostatic Sensor for Human Activity Monitoring.

Identification of a Person in a Trajectory Based on Wearable Sensor Data Analysis

Research on the Human Motion Recognition Method Based on Wearable

A Modular Approach for Synchronized Wireless Multimodal Multisensor Data Acquisition in Highly Dynamic Social Settings

A Multi-Featured Analysis for Body Sensor Networks-based Affective Actions Recognition.

Realtime Interpersonal Human Synchrony Detection Based on Action Segmentation

Human unintentional and intentional interpersonal coordination in interaction with a humanoid robot

Predicting Trust Using Automated Assessment of Multivariate Interactional Synchrony

Computational Social Dynamics: Analyzing the Face-level Interactions in a Group

Human Motion Capture and Recognition Based on Sparse Inertial Sensor.

Dyadic Movement Synchrony Estimation Under Privacy-preserving Conditions