Durable molecular response despite F317L and E255K mutations: Successful treatment of chronic myeloid leukemia with sequential imatinib, nilotinib and dasatinib.

T. Intermesoli,F. Castagnetti,S. Soverini,A. Bussini,O. Spinelli,A. Gnani,R. Bassan,G. Rosti

DOI: https://doi.org/10.1016/j.leukres.2011.09.002

Abstract:

What problem does this paper attempt to address?

Transformer for Ultrafast Ultrasound Localization Microscopy

Gaobo Zhang,Yaoting Yue,Fei Dai,Xin Liu,Dean Ta

DOI: https://doi.org/10.1109/ius54386.2022.9958335

2022-01-01

Abstract:Ultrasound localization microscopy (ULM), which allows for the detection of subwavelength acoustic scattering of intravenously injected microbubbles, enabling non-invasive visualization of microvessels on a micron scale. Unfortunately, the implementation of ultrafast ULM is still constrained by the trade-off between data acquisition time, data processing efficiency and image spatial resolution. More recently, deep learning-based methods (DL-ULM) have exhibited promising potential for accelerating ULM imaging by reducing the number of ultrasound (US) image data ( $L$ frames) required due to the ability to localize to as many microbubbles as possible. Nevertheless, this requires the accumulation of a certain number of microbubble events to obtain the desired super resolution image, which results in data acquisition times of typically tens of seconds. To facilitate ultrafast ULM imaging, in this article, we present a novel netwok, termed as Trans-ULM. Specifically, a multi-branch CNN combined with a recursive transformer to reconstruct a super-resolved image directly from a temporal mean low-resolution image generated by averaging $l$ -frame raw US images with $l$ being considerably smaller than $L$ . The experimental results indicate that, trans-ULM can significantly reduce imaging time as compared to the recently reported ULM-DL method, while maintaining spatial resolution.
Deep Learning for Ultrasound Localization Microscopy

Xin Liu,Tianyang Zhou,Mengyang Lu,Yi Yang,Qiong He,Jianwen Luo

DOI: https://doi.org/10.1109/tmi.2020.2986781

IF: 10.6

2020-10-01

IEEE Transactions on Medical Imaging

Abstract:By localizing microbubbles (MBs) in the vasculature, ultrasound localization microscopy (ULM) has recently been proposed, which greatly improves the spatial resolution of ultrasound (US) imaging and will be helpful for clinical diagnosis. Nevertheless, several challenges remain in fast ULM imaging. The main problems are that current localization methods used to implement fast ULM imaging, e.g., a previously reported localization method based on sparse recovery (CS-ULM), suffer from long data-processing time and exhaustive parameter tuning (optimization). To address these problems, in this paper, we propose a ULM method based on deep learning, which is achieved by using a modified sub-pixel convolutional neural network (CNN), termed as mSPCN-ULM. Simulations and in vivo experiments are performed to evaluate the performance of mSPCN-ULM. Simulation results show that even if under high-density condition (6.4 MBs/mm<sup>2</sup>), a high localization precision ( $sim 28mu text{m}$ in the lateral direction and $sim 24mu text{m}$ in the axial direction) and a high localization reliability (Jaccard index of 0.66) can be obtained by mSPCN-ULM, compared to CS-ULM. The in vivo experimental results indicate that with plane wave scan at a transmit center frequency of 15.625 MHz, microvessels with diameters of $sim 17mu text{m}$ can be detected and adjacent microvessels with a distance of $sim 42mu text{m}$ can be separated. Furthermore, when using GPU acceleration, the data-processing time of mSPCN-ULM can be shortened to ~6 sec/frame in the simulations and ~23 sec/frame in the in vivo experim-nts, which is 3–4 orders of magnitude faster than CS-ULM. Finally, once the network is trained, mSPCN-ULM does not need parameter tuning to implement ULM. As a result, mSPCN-ULM opens the door to implement ULM with fast data-processing speed, high imaging accuracy, short data-acquisition time, and high flexibility (robustness to parameters) characteristics.

engineering, biomedical,imaging science & photographic technology, electrical & electronic,computer science, interdisciplinary applications,radiology, nuclear medicine & medical imaging
Untrained Neural Network Enabling Fast and Universal Structured-Illumination Microscopy

Zitong Ye,Xiaoyan Li,Yile Sun,Yuran Huang,Xu Liu,Yubing Han,Cuifang Kuang

DOI: https://doi.org/10.1364/ol.511983

IF: 3.6

2024-01-01

Optics Letters

Abstract:Structured-illumination microscopy (SIM) offers a twofold resolution enhancement beyond the optical diffraction limit. At present, SIM requires several raw structured-illumination (SI) frames to reconstruct a super-resolution (SR) image, especially the time-consuming reconstruction of speckle SIM, which requires hundreds of SI frames. Considering this, we herein propose an untrained structured-illumination reconstruction neural network (USRNN) with known illumination patterns to reduce the amount of raw data that is required for speckle SIM reconstruction by 20 times and thus improve its temporal resolution. Benefiting from the unsupervised optimizing strategy and CNNs’ structure priors, the high-frequency information is obtained from the network without the requirement of datasets; as a result, a high-fidelity SR image with approximately twofold resolution enhancement can be reconstructed using five frames or less. Experiments on reconstructing non-biological and biological samples demonstrate the high-speed and high-universality capabilities of our method.
Deep learning for fast super-resolution ultrasound microvessel imaging

Shunyao Luan,Xiangyang Yu,Shuang Lei,Chi Ma,Xiao Wang,Xudong Xue,Yi Ding,Teng Ma,Benpeng Zhu

DOI: https://doi.org/10.1088/1361-6560/ad0a5a

IF: 3.5

2023-11-08

Physics in Medicine and Biology

Abstract:Objective: Ultrasound localization microscopy (ULM) enables microvascular reconstruction by localizing microbubbles (MBs). Although ULM can obtain microvascular images that are beyond the ultimate resolution of the ultrasound (US) diffraction limit, it requires long data processing time, and the imaging accuracy is susceptible to the density of MBs. Deep learning (DL)-based ULM is proposed to alleviate these limitations, which simulated MBs at low-resolution and mapped them to coordinates at high-resolution by centroid localization. However, traditional DL-based ULMs are imprecise and computationally complex. Also, the performance of DL is highly dependent on the training datasets, which are difficult to realistically simulate. Approach: A novel architecture called Adaptive Matching Network (AM-Net) and a dataset generation method named multi-mapping (MMP) was proposed to overcome the above challenges. The imaging performance and processing time of the AM-Net have been assessed by simulation and in vivo experiments. Main results: Simulation results show that at high density (20 MBs/frame), when compared to other DL-based ULM, AM-Net achieves higher localization accuracy in the lateral/axial direction. In vivo experiment results show that the AM-Net can reconstruct ~24.3 μm diameter micro-vessels and separate two ~28.3 μm diameter micro-vessels. Furthermore, when processing a 128*128 pixels image in simulation experiments and an 896*1280 pixels image in vivo experiment, the processing time of AM-Net is ~13 sec and ~33 sec, respectively, which are 0.3-0.4 orders of magnitude faster than other DL-based ULM. Significance: We proposes a promising solution for ULM with low computing costs and high imaging performance.

engineering, biomedical,radiology, nuclear medicine & medical imaging
In vivo ultrasound localization microscopy for high-density microbubbles

Gaobo Zhang,Xing Hu,Xuan Ren,Boqian Zhou,Boyi Li,Yifang Li,Jianwen Luo,Xin Liu,Dean Ta

DOI: https://doi.org/10.1016/j.ultras.2024.107410

IF: 4.2

Ultrasonics

Abstract:Ultrasound Localization Microscopy (ULM) surpasses the constraints imposed by acoustic diffraction, achieving sub-wavelength resolution visualization of microvasculature through the precise localization of minute microbubbles (MBs). Nonetheless, the analysis of densely populated regions with overlapping MB point spread responses introduces significant localization errors, limiting the use of technique to low-concentration conditions. This raises a trade-off issue between localization efficiency and MB density. In this work, we present a new deep learning framework that combines Transformer and U-Net architectures, termed ULM-TransUNet. As a non-linear model, it is able to learn the complex data patterns of overlapping MBs in dense conditions for accurate localization. To evaluate the performance of ULM-TransUNet, a series of numerical simulations and in vivo experiments are carried out. Numerical simulation results indicate that ULM-TransUNet achieves high-quality ULM imaging, with improvements of 21.93 % in detection rate, 17.36 % in detection precision, and 20.53 % in detection sensitivity, compared to previous state-of-the-art deep learning (DL) method (e.g., ULM-UNet). For the in vivo experiments, ULM-TransUNet achieves the highest spatial resolution (9.4 μm) and rapid inference speed (26.04 ms/frame). Furthermore, it consistently detects more small vessels and resolves closely spaced vessels more effectively. The outcomes of this work imply that ULM-TransUNet can potentially enhance the microvascular imaging performance on high-density MB conditions.
Ultrafast Ultrasound Localization Microscopy by Conditional Generative Adversarial Network.

Wenting Gu,Boyi Li,Jianwen Luo,Zhuangzhi Yan,Dean Ta,Xin Liu

DOI: https://doi.org/10.1109/tuffc.2022.3222534

2023-01-01

Abstract:Ultrasound localization microscopy (ULM) overcomes the acoustic diffraction limit and enables the visualization of microvasculature at subwavelength resolution. However, challenges remain in ultrafast ULM implementation, where short data acquisition time, efficient data processing speed, and high imaging resolution need to be considered simultaneously. Recently, deep learning (DL)-based methods have exhibited potential in speeding up ULM imaging. Nevertheless, a certain number of ultrasound (US) data ( ${L}$ frames) are still required to accumulate enough localized microbubble (MB) events, leading to an acquisition time within a time span of tens of seconds. To further speed up ULM imaging, in this article, we present a new DL-based method, termed as ULM-GAN. By using a modified conditional generative adversarial network (cGAN) framework, ULM-GAN is able to reconstruct a superresolution image directly from a temporal mean low-resolution (LR) image generated by averaging ${l}$ -frame raw US images with ${l}$ being significantly smaller than ${L}$ . To evaluate the performance of ULM-GAN, a series of numerical simulations and phantom experiments are both implemented. The results of the numerical simulations demonstrate that when performing ULM imaging, ULM-GAN allows $\sim 40$ -fold reduction in data acquisition time and $\sim 61$ -fold reduction in computational time compared with the conventional Gaussian fitting method, without compromising spatial resolution according to the resolution scaled error (RSE). For the phantom experiments, ULM-GAN offers an implementation of ULM with ultrafast data acquisition time ( $\sim 0.33$ s) and ultrafast data processing speed ( $\sim 0.60$ s) that makes it promising to observe rapid biological activities in vivo.
A Novel Method for Ultrasound Localization Microscopy Based on Temporal Correlation Through Deep Learning

Gaobo Zhang,Boyi Li,Xin Liu,Dean Ta

DOI: https://doi.org/10.1109/IUS51837.2023.10307316

2023-01-01

Abstract:Ultrasound localization microscopy (ULM) overcomes the acoustic diffraction limit by localizing tiny microbubbles (MBs), thus enabling the microvascular to be rendered at sub-wavelength resolution. However, current techniques (both traditional and deep learning-based methods) for ULM imaging are limited to processing data from the US frame plane (x-y) for tasks (e.g., localization and tracking), with potential information along the acquisition time t-direction (between adjacent frames) remaining unexplored. In this paper, we propose a novel ULM method based on temporal correlation, termed as TC-ULM. Specifically, we exploit the potential correlations inherent in the y-t plane to further reconstruct the ULM image by extracting the variation features of MBs in the t-direction through a deep learning framework. Numerical simulation results and in vivo experiments indicate that MBs have certain continuity and local correlation between adjacent frames in the y-t plane, thus enabling the reconstruction of high-quality ULM images. At the same time, we also found that compared to previous x-y plane processing methods, TC-ULM can reduce the number of frames required for ULM imaging without sacrificing spatial resolution. This implies that CT-ULM, a novel method, can accelerate ULM imaging to some extent.
Fast super-resolution ultrasound microvessel imaging using spatiotemporal data with deep fully convolutional neural network

U-Wai Lok,Chengwu Huang,Ping Gong,Shanshan Tang,Lulu Yang,Wei Zhang,Yohan Kim,Panagiotis Korfiatis,Daniel J Blezek,Fabrice Lucien,Rongqin Zheng,Joshua D Trzasko,Shigao Chen

DOI: https://doi.org/10.1088/1361-6560/abeb31

IF: 3.5

2021-03-23

Physics in Medicine and Biology

Abstract:Abstract Ultrasound localization microscopy (ULM) has been proposed to image microvasculature beyond the ultrasound diffraction limit. Although ULM can attain microvascular images with a sub-diffraction resolution, long data acquisition time and processing time are the critical limitations. Deep learning-based ULM (deep-ULM) has been proposed to mitigate these limitations. However, microbubble (MB) localization used in deep-ULMs is currently based on spatial information without the use of temporal information. The highly spatiotemporally coherent MB signals provide a strong feature that can be used to differentiate MB signals from background artifacts. In this study, a deep neural network was employed and trained with spatiotemporal ultrasound datasets to better identify the MB signals by leveraging both the spatial and temporal information of the MB signals. Training, validation and testing datasets were acquired from MB suspension to mimic the realistic intensity-varying and moving MB signals. The performance of the proposed network was first demonstrated in the chicken embryo chorioallantoic membrane dataset with an optical microscopic image as the reference standard. Substantial improvement in spatial resolution was shown for the reconstructed super-resolved images compared with power Doppler images. The full-width-half-maximum (FWHM) of a microvessel was improved from 133 μ m to 35 μ m, which is smaller than the ultrasound wavelength (73 μ m). The proposed method was further tested in an in vivo human liver data. Results showed the reconstructed super-resolved images could resolve a microvessel of nearly 170 μ m (FWHM). Adjacent microvessels with a distance of 670 μ m, which cannot be resolved with power Doppler imaging, can be well-separated with the proposed method. Improved contrast ratios using the proposed method were shown compared with that of the conventional deep-ULM method. Additionally, the processing time to reconstruct a high-resolution ultrasound frame with an image size of 1024 × 512 pixels was around 16 ms, comparable to state-of-the-art deep-ULMs.

engineering, biomedical,radiology, nuclear medicine & medical imaging
A Deep Learning Method for Reduction of Microbubble Accumulation Time in Ultrasound Localization Microscopy

Jingke Zhang,Xi Zhang,Yi Yang,Qiong He,Jianwen Luo

DOI: https://doi.org/10.1109/ius46767.2020.9251475

2020-01-01

IUS

Abstract:Ultrasound localization microscopy (ULM) enables unprecedented subwavelength spatial resolution by localizing the ultrasound contrast microbubbles (MBs). In conventional ULM methods, low-concentration MB solutions are used to ensure that the MBs are sufficiently sparse to enable localization of individual MB. This leads to a long data-acquisition time (i.e., microbubble-accumulation time) to accumulate enough MBs events using plane wave (PW) imaging. Considering the continuity of the microvessels and redundancy of MB tracks, in this study, we present a method that leverages deep neural network (DNN) to predict microvasculature using significantly reduced data-acquisition time and achieve quality comparable to that obtained using full data-acquisition time. Phantom experiments show that the proposed method significantly improves the density of localized MBs compared with the conventional method. Besides, in vivo experiments demonstrate that the proposed method is capable of reconstructing ULM images using reduced data-acquisition time with small difference, compared with the conventional method.
Qulm-Dl: Quantitative Ultrasound Localization Microscopy Via Deep Learning

Tianyang Zhou,Mengyang Lu,Yi Yang,Qiong He,Jianwen Luo,Xin Liu

DOI: https://doi.org/10.1109/ius46767.2020.9251447

2020-01-01

Abstract:Ultrasound localization microscopy (ULM) has been developed in recent years to significantly improve the spatial resolution of ultrasound imaging by localizing the microbubbles (MBs) flowing in microvasculature. Nevertheless, challenges remain in ULM. In our previous work (IEEE Trans. Med. Imaging, 10.1109/TMI.2020.2986781), by using a modified sub-pixel convolutional neural network (CNN), we have implemented ULM with fast data-processing speed, high imaging accuracy, short data-acquisition time, and high flexibility characteristics. However, the signal intensities of MBs are not quantitatively recovered, as similar to many other ULM techniques. To overcome this limitation, in this work, we propose a new deep-learning (DL) method, termed as qULM-DL, to recover the true distributions (positions and intensities) of the MBs in ULM. A series of numerical simulations and in vivo experiment are performed to evaluate the performance of qULM-DL. The results show that qULM-DL can not only super-resolve the complex structures but also quantitatively recover MBs intensities in these structures.
Accelerating Super-Resolution Ultrasound Localization Microscopy Using Generative Adversarial Net

Sui Yi-Hui,Guo Xing-Yi,Yu Jun-Jin,Alexander A. Solovev,Ta De-An,Xu Kai-Liang

DOI: https://doi.org/10.7498/aps.71.20220954

2022-01-01

Abstract:Ultrafast ultrasound localization microscopy (uULM) has broken through the fundamental acoustic diffraction limit by accumulating thousands of sub-wavelength microbubble localisation points and improved the spatial resolution by more than one order of magnitude, which is conducive to clinical diagnosis. By localizing individually injected microbubbles and tracking their movement with a subwavelength resolution, the vasculature microscopy can be achieved with micrometer scale. However, the reconstruction of a uULM image often requires tens or even hundreds of seconds of continuous long-range image acquisition, which limits its clinical application. In order to solve this problem, a generative adversarial network (GAN) based deep learning method is proposed to reconstruct the super-resolution ultrasound localization microscopy. In vivo uULM ultrasound datasets are used to train the network to reconstruct dense vascular networks via localized microbubbles. This approach is validated by using another in-vivo dataset obtained in a rat brain. Results show that GAN based ultrafast ultrasound localization microscopy (GAN-uULM) can resolve micro vessels smaller than 10 μm. Besides, GAN-uULM is able to distinguish small vessels that cannot be continuously reconstructed by using a standard uULM reconstruction method. Saturation parameter based on counting the number of explored pixels is used to evaluate the reconstruction quality. The proposed reconstruction approach reduces the data requirement by half and thus significantly accelerates the uULM imaging. It is illustrasted that for a dataset of 292 s ultrafast acquisition, the saturation of standard uULM image is 33%, while that of GAN-uULM can reach 46%. Fourier ring correlation (FRC) method is utilized to measure the spatial resolution in uULM. Resolutions of the images obtained by standard uULM and GAN-ULM are 7.8 μm and 8.9 μm, respectively.In conclusion, the developed deep learning model is able to connect trajectories with less computational complexity and avoids manual tuning and trajectory screening, providing an effective solution for accelerating ultrasound localization microscopy.
Faster super-resolution ultrasound imaging with a deep learning model for tissue decluttering and contrast agent localization

Katherine G Brown,Scott Chase Waggener,Arthur David Redfern,Kenneth Hoyt

DOI: https://doi.org/10.1088/2057-1976/ac2f71

2021-10-25

Abstract:Super-resolution ultrasound (SR-US) imaging allows visualization of microvascular structures as small as tens of micrometers in diameter. However, use in the clinical setting has been impeded in part by ultrasound (US) acquisition times exceeding a breath-hold and by the need for extensive offline computation. Deep learning techniques have been shown to be effective in modeling the two more computationally intensive steps of microbubble (MB) contrast agent detection and localization. Performance gains by deep networks over conventional methods are more than two orders of magnitude and in addition the networks can localize overlapping MBs. The ability to separate overlapping MBs allows use of higher contrast agent concentrations and reduces US image acquisition time. Herein we propose a fully convolutional neural network (CNN) architecture to perform the operations of MB detection as well as localization in a single model. Termed SRUSnet, the network is based on the MobileNetV3 architecture modified for 3-D input data, minimal convergence time, and high-resolution data output using a flexible regression head. Also, we propose to combine linear B-mode US imaging and nonlinear contrast pulse sequencing (CPS) which has been shown to increase MB detection and further reduce the US image acquisition time. The network was trained with in silico data and tested on in vitro data from a tissue-mimicking flow phantom, and on in vivo data from the rat hind limb (N = 3). Images were collected with a programmable US system (Vantage 256, Verasonics Inc., Kirkland, WA) using an L11–4v linear array transducer. The network exceeded 99.9% detection accuracy on in silico data. The average localization accuracy was smaller than the resolution of a pixel (i.e. ). The average processing time on a Nvidia GeForce 2080Ti GPU was 64.5 ms for a 128 × 128-pixel image.
A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy

Léo Milecki,Jonathan Porée,Hatim Belgharbi,Chloé Bourquin,Rafat Damseh,Patrick Delafontaine-Martel,Frédéric Lesage,Maxime Gasse,Jean Provost

DOI: https://doi.org/10.48550/arXiv.2310.08143

2023-10-12

Computer Vision and Pattern Recognition

Abstract:Ultrasound Localization Microscopy can resolve the microvascular bed down to a few micrometers. To achieve such performance microbubble contrast agents must perfuse the entire microvascular network. Microbubbles are then located individually and tracked over time to sample individual vessels, typically over hundreds of thousands of images. To overcome the fundamental limit of diffraction and achieve a dense reconstruction of the network, low microbubble concentrations must be used, which lead to acquisitions lasting several minutes. Conventional processing pipelines are currently unable to deal with interference from multiple nearby microbubbles, further reducing achievable concentrations. This work overcomes this problem by proposing a Deep Learning approach to recover dense vascular networks from ultrasound acquisitions with high microbubble concentrations. A realistic mouse brain microvascular network, segmented from 2-photon microscopy, was used to train a three-dimensional convolutional neural network based on a V-net architecture. Ultrasound data sets from multiple microbubbles flowing through the microvascular network were simulated and used as ground truth to train the 3D CNN to track microbubbles. The 3D-CNN approach was validated in silico using a subset of the data and in vivo on a rat brain acquisition. In silico, the CNN reconstructed vascular networks with higher precision (81%) than a conventional ULM framework (70%). In vivo, the CNN could resolve micro vessels as small as 10 $\mu$m with an increase in resolution when compared against a conventional approach.
Super-Resolved Microbubble Localization in Single-Channel Ultrasound RF Signals Using Deep Learning

Nathan Blanken,Jelmer M. Wolterink,Hervé Delingette,Christoph Brune,Michel Versluis,Guillaume Lajoinie

DOI: https://doi.org/10.48550/arXiv.2204.04537

IF: 4.506

2022-04-09

Medical Physics

Abstract:Recently, super-resolution ultrasound imaging with ultrasound localization microscopy (ULM) has received much attention. However, ULM relies on low concentrations of microbubbles in the blood vessels, ultimately resulting in long acquisition times. Here, we present an alternative super-resolution approach, based on direct deconvolution of single-channel ultrasound radio-frequency (RF) signals with a one-dimensional dilated convolutional neural network (CNN). This work focuses on low-frequency ultrasound (1.7 MHz) for deep imaging (10 cm) of a dense cloud of monodisperse microbubbles (up to 1000 microbubbles in the measurement volume, corresponding to an average echo overlap of 94%). Data are generated with a simulator that uses a large range of acoustic pressures (5-250 kPa) and captures the full, nonlinear response of resonant, lipid-coated microbubbles. The network is trained with a novel dual-loss function, which features elements of both a classification loss and a regression loss and improves the detection-localization characteristics of the output. Whereas imposing a localization tolerance of 0 yields poor detection metrics, imposing a localization tolerance corresponding to 4% of the wavelength yields a precision and recall of both 0.90. Furthermore, the detection improves with increasing acoustic pressure and deteriorates with increasing microbubble density. The potential of the presented approach to super-resolution ultrasound imaging is demonstrated with a delay-and-sum reconstruction with deconvolved element data. The resulting image shows an order-of-magnitude gain in axial resolution compared to a delay-and-sum reconstruction with unprocessed element data.
Short Acquisition Time Super-Resolution Ultrasound Microvessel Imaging via Microbubble Separation

Chengwu Huang,Matthew R. Lowerison,Joshua D. Trzasko,Armando Manduca,Yoram Bresler,Shanshan Tang,Ping Gong,U-Wai Lok,Pengfei Song,Shigao Chen

DOI: https://doi.org/10.1038/s41598-020-62898-9

IF: 4.6

2020-04-07

Scientific Reports

Abstract:Abstract Super-resolution ultrasound localization microscopy (ULM), based on localization and tracking of individual microbubbles (MBs), offers unprecedented microvascular imaging resolution at clinically relevant penetration depths. However, ULM is currently limited by the requirement of dilute MB concentrations to ensure spatially sparse MB events for accurate localization and tracking. The corresponding long imaging acquisition times (tens of seconds or several minutes) to accumulate sufficient isolated MB events for full reconstruction of microvasculature preclude the clinical translation of the technique. To break this fundamental tradeoff between acquisition time and MB concentration, in this paper we propose to separate spatially overlapping MB events into sub-populations, each with sparser MB concentration, based on spatiotemporal differences in the flow dynamics (flow speeds and directions). MB localization and tracking are performed for each sub-population separately, permitting more robust ULM imaging of high-concentration MB injections. The superiority of the proposed MB separation technique over conventional ULM processing is demonstrated in flow channel phantom data, and in the chorioallantoic membrane of chicken embryos with optical imaging as an in vivo reference standard. Substantial improvement of ULM is further demonstrated on a chicken embryo tumor xenograft model and a chicken brain, showing both morphological and functional microvasculature details at super-resolution within a short acquisition time (several seconds). The proposed technique allows more robust MB localization and tracking at relatively high MB concentrations, alleviating the need for dilute MB injections, and thereby shortening the acquisition time of ULM imaging and showing great potential for clinical translation.

multidisciplinary sciences
Deep learning for fast denoising filtering in ultrasound localization microscopy

Xiangyang Yu,Shunyao Luan,Shuang Lei,Jing Huang,Zeqing Liu,Xudong Xue,Teng Ma,Yi Ding,Benpeng Zhu

DOI: https://doi.org/10.1088/1361-6560/acf98f

IF: 3.5

2023-09-15

Physics in Medicine and Biology

Abstract:Super-resolution ultrasound (SR-US) microvascular imaging has important implications in the diagnosis of tumor growth and metastasis. Addition of a denoising filter step in ultrasound localization microscopy (ULM) imaging has been shown to effectively reduce the error localizations of microbubbles (MBs) and improve the resolution of SR-US imaging. However, previous image-denoising methods (e.g., block-matching 3-D, BM3D), required long data acquisition and processing times,which made ULM only able to be processed offline. To overcome this limitation, we proposed deep learning (DL) denoising based on contrastive semi-supervised network (CS-Net). For in vitro flow phantom experiment, the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of single microbubble image were 26.9051 dB and 4.0109 dB. For in vivo experiment of New Zealand rabbit tumor, the SNR and CNR were 12.2921 dB and 6.0554 dB. In addition, single microvessel of 25 μm and two microvessels separated by 44.5 μm could be clearly displayed. In terms of denoising speed, the DL denoising speeds for in vitro and in vivo experiments were 0.041 s/frame and 0.062 s/frame, respectively. Experimental results show that DL denoising can improve the resolution of SR-US while reducing denoising time, thereby making further contributions to the clinical real-time imaging of ULM.

engineering, biomedical,radiology, nuclear medicine & medical imaging
Localization with deep learning networks for super-resolution ultrasound imaging

Katherine Brown,Arthur D. Redfern

DOI: https://doi.org/10.1121/10.0018773

2023-03-01

The Journal of the Acoustical Society of America

Abstract:OBJECTIVE: The objective was to optimize an acquisition methodology and deep learning network to localize a microbubble (MB) contrast agent on a preclinical ultrasound (US) platform with the goal of moving super-resolution ultrasound (SRUS) imaging towards clinical adoption. METHODS: A deep learning network was optimized based on the latest advances in computer vision, convolutional neural networks, and transformer architectures. Synthetic data were produced in an US simulation of tissue with MB at various concentrations flowing in a vascular model. The network was programmed in PyTorch and trained on 12 000 synthetic images. US data were collected with a Vevo 3100 (FUJIFILM VisualSonics Inc) using an MX201 linear transducer. In vivo testing images were obtained from hepatocellular carcinoma (HCC) rat liver tumors. After MB injection (Definity), 3000 frames were collected at 90 Hz. RESULTS: The SRUS in vivo results reveal a high level of detail. The smallest vessel measured 34 μm in diameter. Compared to conventional methods, the network improved performance by 10x on a CPU. A GPU platform could give an additional boost, by as much 100x. CONCLUSIONS: Deep networks for localization show potential for improving performance of SR-US towards a real-time imaging modality. The use of a pre-clinical focused US platform demonstrates that localization can improve the visualization of vascular detail and aid clinical understanding.

acoustics,audiology & speech-language pathology
GAN-Based Ultrasound Localization Microscopy

Wenting Gu,Zhuangzhi Yan,Boyi Li,Chengcheng Liu,Dean Ta,Xin Liu

DOI: https://doi.org/10.1109/IUS54386.2022.9957520

2022-01-01

Abstract:Ultrasound localization microscopy (ULM) breaks the acoustics diffraction limit and allows the imaging of the microvasculature within organs and tumors at sub-wavelength resolution while maintaining the imaging penetration. However, the reconstruction quality of localization-based methods highly depends on sufficient ultrasound data frames with sparsely distributed microbubbles (MBs) in each frame which results in great data storage burden and low processing speed. Here, we proposed a novel method based on generative adversarial networks (GANs) to implement the MB localization in ULM imaging to accelerate the data processing speed. The synthetic results indicate that the proposed method performs well in MB localization task with great robustness to overlapping MBs and achieves higher localization speed once the network be well trained.
Deep Learning for Super-resolution Ultrasound Imaging with Spatiotemporal Data

Arthur David Redfern,Katherine G. Brown

2024-08-03

Abstract:Super-resolution ultrasound imaging (SRUS) is an active area of research as it brings up to a ten-fold improvement in the resolution of microvascular structures. The limitations to the clinical adoption of SRUS include long acquisition times and long image processing times. Both these limitations can be alleviated with deep learning approaches to the processing of SRUS images. In this study we propose an optimized architecture based on modern improvements to convolutional neural networks from the ConvNeXt architecture and further customize the choice of features to improve performance on the specific tasks of both MB detection and localization within a single network. We employ a spatiotemporal input of up to five successive image frames to increase the number of MBs detected. The output structure produces three classifications: a MB detection Boolean for each pixel in the central image frame, as well as x and z offsets at 4-fold subpixel resolution for each MB detected. Ultrasound simulations generated images based on the L22-14v transducer (Verasonics) for training and testing of the proposed SRUS-ConvNeXt network. In vivo image data of a mouse brain was used as further validation of the architecture. The proposed network had the highest performance as measured by F1 score when configured for a 3-frame spatiotemporal input. The smallest localization error of {\lambda}/22 was achieved when the network was configured for a single input frame. The flexibility of the proposed architecture allows extension to 10-fold upscaling for SRUS images with a much lower impact to number of parameters and subsequent increase in inference time than typical U-Net style approaches. This network is promising in the quest to develop a SRUS deep network architecture for real time image formation.

Image and Video Processing
Contrast free super-resolution microvessel imaging based on deep learning

Qi You,Matthew R. Lowerison,YiRang Shin,Nathiya V. ChandraSekaran,Xi Chen,Zhijie Dong,Daniel A. Llano,Mark Anastasio,Pengfei Song

DOI: https://doi.org/10.1121/10.0015725

2022-10-01

The Journal of the Acoustical Society of America

Abstract:Super-resolution microvascular imaging (SRMI) based on ultrasound localization microscopy (ULM) breaks the diffraction limit of conventional ultrasound by exploiting intravascular microbubbles. However, microbubble injection makes ULM not entirely noninvasive and less practical. On the other hand, power Doppler (PD) is contrast-free and widely available in clinic, but PD’s spatial resolution falls short of revealing tissue microvasculature. In this study, we propose a deep learning-based, contrast-free SRMI method to improve the spatial resolution of PD. We designed a modified U-Net structure with 4 layers in depth with 64 to 512 channels. Ultrafast ultrasound data (1000 Hz frame rate) acquired from 16 mice were made into the 4080 data blocks used for training. Reconstructed ULM images using 32 s acquisition served as the labelled ground truth. Contrast-free ultrasound data using 0.4 s acquisition were used as the network input. The results have shown that the proposed DL-based method can significantly improve the spatial resolution of contrast-free power Doppler without creating fake vessels. The full width half-maximum (FWHM) of the cross-sectional profile of a microvessel was improved from 66 to 24 μm. Robust generalizability of the proposed DL method was also demonstrated on various tissues and organs that were absent in training.

acoustics,audiology & speech-language pathology

Durable molecular response despite F317L and E255K mutations: Successful treatment of chronic myeloid leukemia with sequential imatinib, nilotinib and dasatinib.

Transformer for Ultrafast Ultrasound Localization Microscopy

Deep Learning for Ultrasound Localization Microscopy

Untrained Neural Network Enabling Fast and Universal Structured-Illumination Microscopy

Deep learning for fast super-resolution ultrasound microvessel imaging

In vivo ultrasound localization microscopy for high-density microbubbles

Ultrafast Ultrasound Localization Microscopy by Conditional Generative Adversarial Network.

A Novel Method for Ultrasound Localization Microscopy Based on Temporal Correlation Through Deep Learning

Fast super-resolution ultrasound microvessel imaging using spatiotemporal data with deep fully convolutional neural network

A Deep Learning Method for Reduction of Microbubble Accumulation Time in Ultrasound Localization Microscopy

Qulm-Dl: Quantitative Ultrasound Localization Microscopy Via Deep Learning

Accelerating Super-Resolution Ultrasound Localization Microscopy Using Generative Adversarial Net

Faster super-resolution ultrasound imaging with a deep learning model for tissue decluttering and contrast agent localization

A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy

Super-Resolved Microbubble Localization in Single-Channel Ultrasound RF Signals Using Deep Learning

Short Acquisition Time Super-Resolution Ultrasound Microvessel Imaging via Microbubble Separation

Deep learning for fast denoising filtering in ultrasound localization microscopy

Localization with deep learning networks for super-resolution ultrasound imaging

GAN-Based Ultrasound Localization Microscopy

Deep Learning for Super-resolution Ultrasound Imaging with Spatiotemporal Data

Contrast free super-resolution microvessel imaging based on deep learning