Abstract:Ultrasound is a widely used imaging modality, yet it is well-known that scanning can be highly operator-dependent and difficult to perform, which limits its wider use in clinical practice. The literature on understanding what makes clinical sonography hard to learn and how sonography varies in the field is sparse, restricted to small-scale studies on the effectiveness of ultrasound training schemes, the role of ultrasound simulation in training, and the effect of introducing scanning guidelines and standards on diagnostic image quality. The Big Data era, and the recent and rapid emergence of machine learning as a more mainstream large-scale data analysis technique, presents a fresh opportunity to study sonography in the field at scale for the first time. Large-scale analysis of video recordings of full-length routine fetal ultrasound scans offers the potential to characterise differences between the scanning proficiency of experts and trainees that would be tedious and time-consuming to do manually due to the vast amounts of data. Such research would be informative to better understand operator clinical workflow when conducting ultrasound scans to support skills training, optimise scan times, and inform building better user-machine interfaces. This paper is to our knowledge the first to address sonography data science, which we consider in the context of second-trimester fetal sonography screening. Specifically, we present a fully-automatic framework to analyse operator clinical workflow solely from full-length routine second-trimester fetal ultrasound scan videos. An ultrasound video dataset containing more than 200 hours of scan recordings was generated for this study. We developed an original deep learning method to temporally segment the ultrasound video into semantically meaningful segments (the video description). The resulting semantic annotation was then used to depict operator clinical workflow (the knowledge representation). Machine learning was applied to the knowledge representation to characterise operator skills and assess operator variability. For video description, our best-performing deep spatio-temporal network shows favourable results in cross-validation (accuracy: 91.7%), statistical analysis (correlation: 0.98, p < 0.05) and retrospective manual validation (accuracy: 76.4%). For knowledge representation of operator clinical workflow, a three-level abstraction scheme consisting of a Subject-specific Timeline Model (STM), Summary of Timeline Features (STF), and an Operator Graph Model (OGM), was introduced that led to a significant decrease in dimensionality and computational complexity compared to raw video data. The workflow representations were learnt to discriminate between operator skills, where a proposed convolutional neural network-based model showed most promising performance (cross-validation accuracy: 98.5%, accuracy on unseen operators: 76.9%). These were further used to derive operator-specific scanning signatures and operator variability in terms of type, order and time distribution of constituent tasks.

Statistical Characterisation of Fetal Anatomy in Simple Obstetric Ultrasound Video Sweeps

A Preliminary Study of Fetal Intelligent Navigation Echocardiography in the Normal Fetuses During Second and Third Trimester

Clinical workflow of sonographers performing fetal anomaly ultrasound scans: deep-learning-based analysis

Fetal growth analysis from ultrasound videos based on different biometrics using optimal segmentation and hybrid classifier

Quantitative Approach to Quality Review of Prenatal Ultrasound Examinations: Fetal Biometry

AutoFB: Automating Fetal Biometry Estimation from Standard Ultrasound Planes

Whole-examination AI estimation of fetal biometrics from 20-week ultrasound scans

Automating the Human Action of First-Trimester Biometry Measurement from Real-World Freehand Ultrasound

Deep learning fetal ultrasound video model match human observers in biometric measurements

Autonomous fetal morphology scan: deep learning + clustering merger - the second pair of eyes behind the doctor

Developing Clinical Artificial Intelligence for Obstetric Ultrasound to Improve Access in Underserved Regions: Protocol for a Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) Study

Artificial Intelligence in Obstetric Anomaly Scan: Heart and Brain

Towards deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via Ultrasound Images

Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos

SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound

Automatic deep learning-based pipeline for automatic delineation and measurement of fetal brain structures in routine mid-trimester ultrasound images

A benchmark for 2D foetal brain ultrasound analysis

FPUS23: An Ultrasound Fetus Phantom Dataset With Deep Neural Network Evaluations for Fetus Orientations, Fetal Planes, and Anatomical Features

FetusMap: Fetal Pose Estimation in 3D Ultrasound

BiometryNet: Landmark-based Fetal Biometry Estimation from Standard Ultrasound Planes