Abstract:Abstract Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates’ body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 × 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications.

RGB-D scene analysis in the NICU

Fast body part segmentation and tracking of neonatal video data using deep learning

Continuous non-contact vital sign monitoring of neonates in intensive care units using RGB-D cameras

Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers

Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks

Hybrid Deep Learning-Based for Enhanced Occlusion Segmentation in PICU Patient Monitoring

Neonatal Face and Facial Landmark Detection from Video Recordings

Accurate prediction of neurologic changes in critically ill infants using pose AI

Non-Contact Heart Rate Monitoring In Neonatal Intensive Care Unit Using Rgb Camera

Conditional Generative Adversarial Networks for Data Augmentation of a Neonatal Image Dataset

Robust Neonatal Face Detection in Real-world Clinical Settings

Accurate Neonatal Face Detection for Improved Pain Classification in the Challenging NICU Setting

Vision-language models for decoding provider attention during neonatal resuscitation

Assessing spectral effectiveness in color fundus photography for deep learning classification of retinopathy of prematurity

Summarization of ICU Patient Motion from Multimodal Multiview Videos

Deep Learning Mixture-of-Experts Approach for Cytotoxic Edema Assessment in Infants and Children

Generalized Camera-Based Infant Sleep-Wake Monitoring in NICUs: A Multi-Center Clinical Trial

Dense captioning and multidimensional evaluations for indoor robotic scenes

Video-Based Respiratory Rate Estimation for Infants in the NICU

Multimodality Video Acquisition System for the Assessment of Vital Distress in Children

Automated Analysis for Retinopathy of Prematurity by Deep Neural Networks