Video-Based Neonatal Pain Assessment in Uncontrolled Conditions.

Huaiyu Zhu,Yisheng Zhao,Xiaofei Chen,Feixiang Luo,Lingli Mei,Shuohui Chen,Yun Pan

DOI: https://doi.org/10.1109/jbhi.2023.3324537

IF: 7.7

2023-01-01

IEEE Journal of Biomedical and Health Informatics

Abstract:OBJECTIVE:Neonatal pain can have long-term adverse effects on newborns' cognitive and neurological development. Video-based Neonatal Pain Assessment (NPA) method has gained increasing attention due to its performance and practicality. However, existing methods focus on assessment under controlled environments while ignoring real-life disturbances present in uncontrolled conditions.METHODS:We propose a video-based NPA method, which is robust to four real-life disturbances and adaptively highlights keyframes. Our method involves a region-channel-attention module for extracting facial features under the disturbances of facial occlusion and pose variation; a body language analysis module robust to disturbances from body occlusion and movement interference, which utilizes skeleton sequences to represent the neonate's body; and a keyframes-aware convolution to get rid of information located at non-contributing moments. For evaluation, we built an NPA video dataset of 1091 neonates with disturbance annotations.RESULTS:The results show that our method consistently outperforms state-of-the-art methods on the full dataset and nine subsets, where it achieves an accuracy of 91.04% on the full dataset with an accuracy increment of 6.27%. Contributions: We present the problem of video-based NPA under uncontrolled conditions, propose a method robust to four disturbances, and construct a video NPA dataset, thus facilitating the practical applications of NPA.