Abstract:Ultrasound is a widely used imaging modality, yet it is well-known that scanning can be highly operator-dependent and difficult to perform, which limits its wider use in clinical practice. The literature on understanding what makes clinical sonography hard to learn and how sonography varies in the field is sparse, restricted to small-scale studies on the effectiveness of ultrasound training schemes, the role of ultrasound simulation in training, and the effect of introducing scanning guidelines and standards on diagnostic image quality. The Big Data era, and the recent and rapid emergence of machine learning as a more mainstream large-scale data analysis technique, presents a fresh opportunity to study sonography in the field at scale for the first time. Large-scale analysis of video recordings of full-length routine fetal ultrasound scans offers the potential to characterise differences between the scanning proficiency of experts and trainees that would be tedious and time-consuming to do manually due to the vast amounts of data. Such research would be informative to better understand operator clinical workflow when conducting ultrasound scans to support skills training, optimise scan times, and inform building better user-machine interfaces. This paper is to our knowledge the first to address sonography data science, which we consider in the context of second-trimester fetal sonography screening. Specifically, we present a fully-automatic framework to analyse operator clinical workflow solely from full-length routine second-trimester fetal ultrasound scan videos. An ultrasound video dataset containing more than 200 hours of scan recordings was generated for this study. We developed an original deep learning method to temporally segment the ultrasound video into semantically meaningful segments (the video description). The resulting semantic annotation was then used to depict operator clinical workflow (the knowledge representation). Machine learning was applied to the knowledge representation to characterise operator skills and assess operator variability. For video description, our best-performing deep spatio-temporal network shows favourable results in cross-validation (accuracy: 91.7%), statistical analysis (correlation: 0.98, p < 0.05) and retrospective manual validation (accuracy: 76.4%). For knowledge representation of operator clinical workflow, a three-level abstraction scheme consisting of a Subject-specific Timeline Model (STM), Summary of Timeline Features (STF), and an Operator Graph Model (OGM), was introduced that led to a significant decrease in dimensionality and computational complexity compared to raw video data. The workflow representations were learnt to discriminate between operator skills, where a proposed convolutional neural network-based model showed most promising performance (cross-validation accuracy: 98.5%, accuracy on unseen operators: 76.9%). These were further used to derive operator-specific scanning signatures and operator variability in terms of type, order and time distribution of constituent tasks.

Whole-examination AI estimation of fetal biometrics from 20-week ultrasound scans

Automating the Human Action of First-Trimester Biometry Measurement from Real-World Freehand Ultrasound

Human-level Performance On Automatic Head Biometrics In Fetal Ultrasound Using Fully Convolutional Neural Networks

AutoFB: Automating Fetal Biometry Estimation from Standard Ultrasound Planes

Deep learning fetal ultrasound video model match human observers in biometric measurements

Enabling faster and more reliable sonographic assessment of gestational age through machine learning

Multi-Task Learning Approach for Unified Biometric Estimation from Fetal Ultrasound Anomaly Scans

SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound

Automatic Estimation of Fetal Abdominal Circumference from Ultrasound Images

Deep learning for estimation of fetal weight throughout the pregnancy from fetal abdominal ultrasound

Fetal biometric based abnormality detection during prenatal development using deep learning techniques

Fetal growth analysis from ultrasound videos based on different biometrics using optimal segmentation and hybrid classifier

BiometryNet: Landmark-based Fetal Biometry Estimation from Standard Ultrasound Planes

FetalNet: Multi-task Deep Learning Framework for Fetal Ultrasound Biometric Measurements

Deep learning to estimate gestational age from fly‐to cineloop videos: A novel approach to ultrasound quality control

A new approach to automatic measure fetal head circumference in ultrasound images using convolutional neural networks

Fetal Ultrasound Image Segmentation for Measuring Biometric Parameters Using Multi-Task Deep Learning

Leveraging Clinically Relevant Biometric Constraints To Supervise A Deep Learning Model For The Accurate Caliper Placement To Obtain Sonographic Measurements Of The Fetal Brain

DeepGA for Automatically Estimating Fetal Gestational Age Through Ultrasound Imaging.

Autonomous fetal morphology scan: deep learning + clustering merger - the second pair of eyes behind the doctor

Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos