Abstract:Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray (CXR) images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced. Through comprehensive experiments, it is shown that networks pertained on CXR images outperform networks pretrained on natural image datasets such as ImageNet. Qualitatively and quantitatively evaluations also reveal that the predictive uncertainty estimates are statistically higher for erroneous predictions than correct predictions. Accordingly, uncertainty quantification methods are capable of flagging risky predictions with high uncertainty estimates. We also observe that ensemble methods more reliably capture uncertainties during the inference. DNN-based solutions for COVID-19 detection have been mainly proposed without any principled mechanism for risk mitigation. Previous studies have mainly focused on on generating single-valued predictions using pretrained DNNs. In this paper, we comprehensively apply and comparatively evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced for the first time. Using these new uncertainty performance metrics, we quantitatively demonstrate when we could trust DNN predictions for COVID-19 detection from chest X-rays. It is important to note the proposed novel uncertainty evaluation metrics are generic and could be applied for evaluation of probabilistic forecasts in all classification problems.

Uncertainty Quantification for Deep Learning-Based Remote Photoplethysmography

HRUNet: Assessing Uncertainty in Heart Rates Measured from Facial Videos

Towards Practical Facial Video-based Remote Heart Rate Estimation Via Cross Domain Rppg Adaptation.

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

A Survey on Uncertainty Quantification Methods for Deep Learning

Robust Remote Photoplethysmography Estimation With Environmental Noise Disentanglement

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study

U-FaceBP: Uncertainty-aware Bayesian Ensemble Deep Learning for Face Video-based Blood Pressure Measurement

Objective evaluation of deep uncertainty predictions for COVID-19 detection

A Compensation Network With Error Mapping for Robust Remote Photoplethysmography in Noise-Heavy Conditions

Idtl-Rppg: Remote Heart Rate Estimation Using Instance-Based Deep Transfer Learning

A Survey on Uncertainty Quantification Methods for Deep Neural Networks: An Uncertainty Source Perspective

Leveraging the Bhattacharyya coefficient for uncertainty quantification in deep neural networks

Remote Heart Rate Estimation by Signal Quality Attention Network

Uncertainty Quantification of Neural Reflectance Fields for Underwater Scenes

Analysis of CNN-based remote-PPG to understand limitations and sensitivities

A LSTM-Based Realtime Signal Quality Assessment for Photoplethysmogram and Remote Photoplethysmogram

Classification of Photoplethysmographic Signal Quality with Deep Convolution Neural Networks for Accurate Measurement of Cardiac Stroke Volume

Quantifying Uncertainty in Deep Learning of Radiologic Images