Abstract:BackgroundThe trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further analyses. Manual inspection and visual QC of each segmentation result is not feasible at large scale. However, it is important to be able to automatically detect when a segmentation method fails in order to avoid inclusion of wrong measurements into subsequent analyses which could otherwise lead to incorrect conclusions.MethodsTo overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4800 cardiovascular magnetic resonance (CMR) scans. We then apply our method to a large cohort of 7250 CMR on which we have performed manual QC.ResultsWe report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using the predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4800 scans for which manual segmentations were available. We mimic real-world application of the method on 7250 CMR where we show good agreement between predicted quality metrics and manual visual QC scores.ConclusionsWe show that Reverse classification accuracy has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study.

Real-time Prediction of Segmentation Quality

Automated 3D Fetal Brain Segmentation Using an Optimized Deep Learning Approach

Quality Control-Driven Image Segmentation Towards Reliable Automatic Image Analysis in Large-Scale Cardiovascular Magnetic Resonance Aortic Cine Imaging

Deep neural network ensemble for on-the-fly quality control-driven segmentation of cardiac MRI T1 mapping

Automated Quality Control in Image Segmentation: Application to the UK Biobank Cardiac MR Imaging Study

Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study

Deep Generative Model-based Quality Control for Cardiac MRI Segmentation

QCResUNet: Joint Subject-level and Voxel-level Segmentation Quality Prediction

TMS-Net: A Segmentation Network Coupled With A Run-time Quality Control Method For Robust Cardiac Image Segmentation

A quality assurance framework for real-time monitoring of deep learning segmentation models in radiotherapy

Assessment of Deep Learning Segmentation for Real-Time Free-Breathing Cardiac Magnetic Resonance Imaging at Rest and Under Exercise Stress

Invited:Hardware-aware Real-time Myocardial Segmentation Quality Control in Contrast Echocardiography

Segmentation quality assessment by automated detection of erroneous surface regions in medical images

Improved Robustness for Deep Learning-based Segmentation of Multi-Center Myocardial Perfusion MRI Datasets Using Data Adaptive Uncertainty-guided Space-time Analysis

Patch-wise 3D segmentation quality assessment combining reconstruction and regression networks

Hardware-aware Real-time Myocardial Segmentation Quality Control in Contrast Echocardiography

Towards Efficient Human-Machine Collaboration: Real-Time Correction Effort Prediction for Ultrasound Data Acquisition

Temporal Uncertainty Localization to Enable Human-in-the-loop Analysis of Dynamic Contrast-enhanced Cardiac MRI Datasets

A Decoupled Uncertainty Model for MRI Segmentation Quality Estimation