Abstract:Abstract Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates’ body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 × 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications.

Depth-Based Intervention Detection in the Neonatal Intensive Care Unit Using Vision Transformers

Use of a Multiscale Vision Transformer to predict Nursing Activities Score from Low Resolution Thermal Videos in an Intensive Care Unit

Hybrid Deep Learning-Based for Enhanced Occlusion Segmentation in PICU Patient Monitoring

Vision-language models for decoding provider attention during neonatal resuscitation

Video object detection for privacy-preserving patient monitoring in intensive care

Enhanced Infant Movement Analysis Using Transformer-Based Fusion of Diverse Video Features For Neurodevelopmental Monitoring

Continuous non-contact vital sign monitoring of neonates in intensive care units using RGB-D cameras

Preterm infants' limb-pose estimation from depth images using convolutional neural networks

Clinical Intervention Prediction and Understanding using Deep Networks

Automated Discomfort Detection for Premature Infants in NICU Using Time-Frequency Feature-Images and CNNs

Deep learning based non-contact physiological monitoring in Neonatal Intensive Care Unit

Privacy-Preserving Action Recognition for Smart Hospitals using Low-Resolution Depth Images

A Multi-center Clinical Trial for Camera-based Infant Sleep and Awake Detection in Neonatal Intensive Care Unit

Neonatal Face and Facial Landmark Detection from Video Recordings

RT-ViT: Real-Time Monocular Depth Estimation Using Lightweight Vision Transformers

Preterm infants' pose estimation with spatio-temporal features

Depth-Weighted Detection of Behaviours of Risk in People with Dementia using Cameras

Multimodality Video Acquisition System for the Assessment of Vital Distress in Children

Robust Neonatal Face Detection in Real-world Clinical Settings

Accurate prediction of neurologic changes in critically ill infants using pose AI

Fast body part segmentation and tracking of neonatal video data using deep learning