LOTUS: Learning to Optimize Task-based US representations

Yordanka Velikova,Mohammad Farid Azampour,Walter Simson,Vanessa Gonzalez Duque,Nassir Navab
2023-07-30
Abstract:Anatomical segmentation of organs in ultrasound images is essential to many clinical applications, particularly for diagnosis and monitoring. Existing deep neural networks require a large amount of labeled data for training in order to achieve clinically acceptable performance. Yet, in ultrasound, due to characteristic properties such as speckle and clutter, it is challenging to obtain accurate segmentation boundaries, and precise pixel-wise labeling of images is highly dependent on the expertise of physicians. In contrast, CT scans have higher resolution and improved contrast, easing organ identification. In this paper, we propose a novel approach for learning to optimize task-based ultra-sound image representations. Given annotated CT segmentation maps as a simulation medium, we model acoustic propagation through tissue via ray-casting to generate ultrasound training data. Our ultrasound simulator is fully differentiable and learns to optimize the parameters for generating physics-based ultrasound images guided by the downstream segmentation task. In addition, we train an image adaptation network between real and simulated images to achieve simultaneous image synthesis and automatic segmentation on US images in an end-to-end training setting. The proposed method is evaluated on aorta and vessel segmentation tasks and shows promising quantitative results. Furthermore, we also conduct qualitative results of optimized image representations on other organs.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
This paper aims to solve the challenging problem of organ segmentation in ultrasound images. Specifically, existing deep neural networks require a large amount of labeled data for training to achieve clinically acceptable performance. However, in ultrasound images, due to characteristics such as speckle noise and cluttered backgrounds, it is very difficult to obtain accurate segmentation boundaries, and pixel - level labeling of images is highly dependent on doctors' professional knowledge. In comparison, CT scans have higher resolution and contrast, making organ identification easier. Therefore, this paper proposes a new method to optimize task - based ultrasound image representation. By using labeled CT segmentation maps as simulation media, it models the propagation of sound waves in tissues to generate ultrasound training data. This ultrasound simulator is fully differentiable and can optimize the parameters for generating physically - based ultrasound images according to downstream segmentation tasks. In addition, an image adaptation network is trained between real and simulated images to achieve image synthesis and automatic ultrasound image segmentation in an end - to - end training setting. This method was evaluated on aortic and vascular segmentation tasks and showed promising quantitative results. At the same time, qualitative results of optimized image representation for other organs were also presented.