Abstract:Medical images are generally acquired with limited field-of-view (FOV), which could lead to incomplete regions of interest (ROI), and thus impose a great challenge on medical image analysis. This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets. Based on learning a navigation policy, instead of predicting targets directly, reinforcement learning (RL)-based methods have the potential to tackle this challenge in an efficient manner. Inspired by this, in this work we propose a multi-agent RL framework for simultaneous multi-target landmark detection. This framework is aimed to learn from incomplete or (and) complete images to form an implicit knowledge of global structure, which is consolidated during the training stage for the detection of targets from either complete or incomplete test images. To further explicitly exploit the global structural information from incomplete images, we propose to embed a shape model into the RL process. With this prior knowledge, the proposed RL model can not only localize dozens of targets simultaneously, but also work effectively and robustly in the presence of incomplete images. We validated the applicability and efficacy of the proposed method on various multi-target detection tasks with incomplete images from practical clinics, using body dual-energy X-ray absorptiometry (DXA), cardiac MRI and head CT datasets. Results showed that our method could predict whole set of landmarks with incomplete training images up to 80% missing proportion (average distance error 2.29 cm on body DXA), and could detect unseen landmarks in regions with missing image information outside FOV of target images (average distance error 6.84 mm on 3D half-head CT). Our code will be released via https://zmiclab.github.io/projects.html .

Contrastive Learning of Relative Position Regression for One-Shot Object Localization in 3D Medical Images

3D Multiple-Contextual ROI-Attention Network for Efficient and Accurate Volumetric Medical Image Segmentation.

GLRP: Global and Local Contrastive Learning Based on Relative Position for Medical Image Segmentation on Cardiac MRI

One-Shot Medical Landmark Localization by Edge-Guided Transform and Noisy Landmark Refinement

One-Shot Medical Landmark Localization by Edge-Guided Transform and Noisy Landmark Refinement

Fast Multiple Landmark Localisation Using a Patch-based Iterative Network

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding

CAMLD: Contrast-Agnostic Medical Landmark Detection with Consistency-Based Regularization

Multi-target landmark detection with incomplete images via reinforcement learning and shape prior embedding

Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentation

Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization

3D RoI-aware U-Net for Accurate and Efficient Colorectal Tumor Segmentation

Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images

Adaptive 3D Localization of 2D Freehand Ultrasound Brain Images

DeepATLAS: One-Shot Localization for Biomedical Data

Learning to Localize in Unseen Scenes with Relative Pose Regressors

Unsupervised Local Discrimination for Medical Images

ConvNet-Based Localization of Anatomical Structures in 3D Medical Images

Spine Landmark Localization with combining of Heatmap Regression and Direct Coordinate Regression

Treatment of severe and mild hypercholesterolaemia with probucol and neomycin..