Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model

Haojun Jiang,Zhenguo Sun,Ning Jia,Meng Li,Yu Sun,Shaqi Luo,Shiji Song,Gao Huang

2024-10-21

Abstract:Echocardiography is the only technique capable of real-time imaging of the heart and is vital for diagnosing the majority of cardiac diseases. However, there is a severe shortage of experienced cardiac sonographers, due to the heart's complex structure and significant operational challenges. To mitigate this situation, we present a Cardiac Copilot system capable of providing real-time probe movement guidance to assist less experienced sonographers in conducting freehand echocardiography. This system can enable non-experts, especially in primary departments and medically underserved areas, to perform cardiac ultrasound examinations, potentially improving global healthcare delivery. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. This world model can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers. Evaluations on three standard planes with 37K sample pairs demonstrate that the world model can reduce navigation errors by up to 33\% and exhibit more stable performance.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The main problem this paper attempts to address is providing real-time probe movement guidance through the development of a system called Cardiac Copilot, to assist less experienced cardiac sonographers in performing handheld cardiac ultrasound examinations. Specifically, the paper aims to:

1. **Alleviate Technician Shortage**: Due to the complexity of cardiac structures and the high difficulty of operations, there is currently a severe shortage of cardiac sonographers, especially in primary healthcare institutions and resource-limited areas. The goal of the Cardiac Copilot system is to enable non-professionals to perform cardiac ultrasound examinations, thereby improving the accessibility of global healthcare services.

2. **Improve Operational Accuracy**: By introducing a data-driven world model—Cardiac Dreamer, the system can represent the spatial structure of the heart in a latent space and provide structural features of any cardiac plane around the current probe position as an accurate navigation map. This helps reduce navigation errors and improve operational stability.

3. **Optimize Training Process**: Traditional training requires at least 12 months of clinical practice, while the Cardiac Copilot system is expected to shorten this process, allowing more people to quickly master the techniques of cardiac ultrasound examination.

In summary, by developing the Cardiac Copilot system, this paper aims to address the issues of technician shortage, operational difficulty, and long training cycles in cardiac ultrasound examinations, thereby enhancing the efficiency and quality of cardiac ultrasound examinations.