Abstract:General movement assessment (GMA) of infant movement videos (IMVs) is an effective method for early detection of cerebral palsy (CP) in infants. We demonstrate in this paper that end-to-end trainable neural networks for image sequence recognition can be applied to achieve good results in GMA, and more importantly, augmenting raw video with infant body parsing and pose estimation information can significantly improve performance. To solve the problem of efficiently utilizing partially labeled IMVs for body parsing, we propose a semi-supervised model, termed SiamParseNet (SPN), which consists of two branches, one for intra-frame body parts segmentation and another for inter-frame label propagation. During training, the two branches are jointly trained by alternating between using input pairs of only labeled frames and input of both labeled and unlabeled frames. We also investigate training data augmentation by proposing a factorized video generative adversarial network (FVGAN) to synthesize novel labeled frames for training. FVGAN decouples foreground and background generation which allows for generating multiple labeled frames from one real labeled frame. When testing, we employ a multi-source inference mechanism, where the final result for a test frame is either obtained via the segmentation branch or via propagation from a nearby key frame. We conduct extensive experiments for body parsing using SPN on two infant movement video datasets; on these partially labeled IMVs, we show that SPN coupled with FVGAN achieves state-of-the-art performance. We further demonstrate that our proposed SPN can be easily adapted to the infant pose estimation task with superior performance. Last but not least, we explore the clinical application of our method for GMA. We collected a new clinical IMV dataset with GMA annotations, and our experiments show that our SPN models for body parsing and pose estimation trained on the first two datasets generalize well to the new clinical dataset and their results can significantly boost the convolutional recurrent neural network (CRNN) based GMA prediction performance when combined with raw video inputs.

Weakly Supervised Online Action Detection for Infant General Movements

Automating General Movements Assessment with Quantitative Deep Learning to Facilitate Early Screening of Cerebral Palsy

Preterm Infant General Movements Assessment Via Representation Learning

Cost-effective Video-based Poor Repertoire Detection for Preterm Infant General Movement Analysis.

RGB-D Videos-Based Early Prediction of Infant Cerebral Palsy Via General Movements Complexity

Semi-supervised body parsing and pose estimation for enhancing infant general movement assessment

Automatic Quantitative Intelligent Assessment of Neonatal General Movements with Video Tracking

Automated Classification of General Movements in Infants Using a Two-stream Spatiotemporal Fusion Network

Automatic Classification of General Movements in Newborns

Automatically Measure the Quality of Infants’ Spontaneous Movement Via Videos to Predict the Risk of Cerebral Palsy

Automated detection of abnormal general movements from pressure and positional information in hospitalized infants

WOAD: Weakly Supervised Online Action Detection in Untrimmed Videos

Deep-learning for automated markerless tracking of infants general movements

A Training-Free Infant Spontaneous Movement Assessment Method for Cerebral Palsy Prediction Based on Videos.

A Spatio-temporal Attention-based Model for Infant Movement Assessment from Videos

Towards Reliable, Automated General Movement Assessment for Perinatal Stroke Screening in Infants Using Wearable Accelerometers

Intelligence Sparse Sensor Network for Automatic Early Evaluation of General Movements in Infants

A self-supervised spatio-temporal attention network for video-based 3D infant pose estimation

Infant movement classification through pressure distribution analysis

A pose estimation for motion tracking of infants cerebral palsy

Detection of Infantile Movement Disorders in Video Data Using Deformable Part-Based Model