Abstract:Abstract Pain intensity serves as an essential indicator of patient discomfort. The analysis of video monitoring for the evaluation of pain intensity has emerged as a significant area of research. Convolutional neural network (CNN)-based spatiotemporal models are extensively utilized to extract dynamic pain information from videos for assessing pain intensity. Nonetheless, challenges persist in effectively extracting spatiotemporal dynamic information, because pain typically appears in specific consecutive frames and within certain regions. Consequently, this research introduces the attention-aware spatiotemporal network (AASTNet) designed for the evaluation of pain intensity, incorporating both temporal and spatial sub-networks. Specifically, the system employs a temporal sub-network, succeeded by a feature gating mechanism, similar to a self-attention mechanism, to selectively focus on video frames within a temporal sequence that contain pertinent pain-related information. Furthermore, an attention mechanism is integrated into the spatial sub-network to emphasize the spatially localized regions closely related to pain. To counteract the issue of spatially global information deficiency induced by the attention mechanism, this method integrates the geometric information of facial landmarks to capture the overall topological patterns of the facial regions associated with pain. The feature vectors derived from the two sub-networks are subsequently combined and utilized for regression analysis to estimate pain intensity. The proposed AASTNet is assessed on UNBC-McMaster Shoulder Pain Expression Archive Database, and the results reveal that the proposed AASTNet outperforms the contemporary state-ofthe-art methods.

A Deep Attention Transformer Network for Pain Estimation with Facial Expression Video

Hierarchical Global and Local Transformer for Pain Estimation with Facial Expression Videos

Pain-attentive Network: a Deep Spatio-Temporal Attention Model for Pain Estimation

Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signals

AASTNet: an Attention-Aware Spatial-Temporal Network for Enhanced Pain Intensity Evaluation in Facial Image

Transformer with Leveraged Masked Autoencoder for video-based Pain Assessment

Two-Stream Attention Network for Pain Recognition from Video Sequences

A Full Transformer-based Framework for Automatic Pain Estimation using Videos

Pain-awareness Multistream Convolutional Neural Network for Pain Estimation.

Pain estimation with integrating global-wise and region-wise convolutional networks

Automatic Pain Estimation from Facial Expressions: A Comparative Analysis Using Off-the-Shelf CNN Architectures

Deep Spatiotemporal Representation of the Face for Automatic Pain Intensity Estimation

Spatio-Temporal Pain Estimation Network With Measuring Pseudo Heart Rate Gain

HybNet: a Hybrid Network Structure for Pain Intensity Estimation

Fusing Deep Learned and Hand-Crafted Features of Appearance, Shape, and Dynamics for Automatic Pain Estimation

DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain

Pain Analysis using Adaptive Hierarchical Spatiotemporal Dynamic Imaging

A Spatiotemporal Convolutional Neural Network for Automatic Pain Intensity Estimation from Facial Dynamics

Global-Local Combined Features to Detect Pain Intensity from Facial Expression Images with Attention Mechanism

Multi-Scale Regional Attention Networks for Pain Estimation

Multimodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures