Abstract:Ultrasound is a widely used imaging modality, yet it is well-known that scanning can be highly operator-dependent and difficult to perform, which limits its wider use in clinical practice. The literature on understanding what makes clinical sonography hard to learn and how sonography varies in the field is sparse, restricted to small-scale studies on the effectiveness of ultrasound training schemes, the role of ultrasound simulation in training, and the effect of introducing scanning guidelines and standards on diagnostic image quality. The Big Data era, and the recent and rapid emergence of machine learning as a more mainstream large-scale data analysis technique, presents a fresh opportunity to study sonography in the field at scale for the first time. Large-scale analysis of video recordings of full-length routine fetal ultrasound scans offers the potential to characterise differences between the scanning proficiency of experts and trainees that would be tedious and time-consuming to do manually due to the vast amounts of data. Such research would be informative to better understand operator clinical workflow when conducting ultrasound scans to support skills training, optimise scan times, and inform building better user-machine interfaces. This paper is to our knowledge the first to address sonography data science, which we consider in the context of second-trimester fetal sonography screening. Specifically, we present a fully-automatic framework to analyse operator clinical workflow solely from full-length routine second-trimester fetal ultrasound scan videos. An ultrasound video dataset containing more than 200 hours of scan recordings was generated for this study. We developed an original deep learning method to temporally segment the ultrasound video into semantically meaningful segments (the video description). The resulting semantic annotation was then used to depict operator clinical workflow (the knowledge representation). Machine learning was applied to the knowledge representation to characterise operator skills and assess operator variability. For video description, our best-performing deep spatio-temporal network shows favourable results in cross-validation (accuracy: 91.7%), statistical analysis (correlation: 0.98, p < 0.05) and retrospective manual validation (accuracy: 76.4%). For knowledge representation of operator clinical workflow, a three-level abstraction scheme consisting of a Subject-specific Timeline Model (STM), Summary of Timeline Features (STF), and an Operator Graph Model (OGM), was introduced that led to a significant decrease in dimensionality and computational complexity compared to raw video data. The workflow representations were learnt to discriminate between operator skills, where a proposed convolutional neural network-based model showed most promising performance (cross-validation accuracy: 98.5%, accuracy on unseen operators: 76.9%). These were further used to derive operator-specific scanning signatures and operator variability in terms of type, order and time distribution of constituent tasks.

Knowledge distillation for efficient standard scanplane detection of fetal ultrasound

SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound

Fetal Ultrasound Standard Plane Detection With Coarse-to-Fine Multi-Task Learning

Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos

The Application of Knowledge Distillation Toward Fine-Grained Segmentation for Three-Vessel View of Fetal Heart Ultrasound Images

Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks

Standard Plane Identification in Fetal Brain Ultrasound Scans Using a Differential Convolutional Neural Network

Automatic Fetal Ultrasound Standard Plane Recognition Based on Deep Learning and IIoT

Deep Learning-Based Methodology for Recognition of Fetal Brain Standard Scan Planes in 2D Ultrasound Images

Towards Standard Plane Prediction of Fetal Head Ultrasound with Domain Adaption

EchoDFKD: Data-Free Knowledge Distillation for Cardiac Ultrasound Segmentation using Synthetic Data

Efficient Feature Extraction Using Light-Weight CNN Attention-Based Deep Learning Architectures for Ultrasound Fetal Plane Classification

Automatic Detection of Standard Planes in Fetal Ultrasound Images based on Convolutional Neural Networks and Ensemble Learning

Measuring proximity to standard planes during fetal brain ultrasound scanning

SPRNet: Automatic Fetal Standard Plane Recognition Network for Ultrasound Images

Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks

Fetal Abdominal Standard Plane Localization through Representation Learning with Knowledge Transfer

A Deep Learning-Based Intelligent Analysis Platform for Fetal Ultrasound Four-Chamber Views

Deep learning fetal ultrasound video model match human observers in biometric measurements

Automated Detection of Congenital Heart Disease in Fetal Ultrasound Screening