Abstract:Purpose: Despite advances in deep learning, robust medical image segmentation in the presence of arti-facts, pathology, and other imaging shortcomings has remained a challenge. In this paper, we demonstrate that by synergistically marrying the unmatched strengths of high-level human knowledge (i.e., natural intelligence (NI)) with the capabilities of deep learning (DL) networks (i.e., artificial intelligence (AI)) in garnering intricate details, these challenges can be significantly overcome. Focusing on the object recog-nition task, we formulate an anatomy-guided deep learning object recognition approach named AAR-DL which combines an advanced anatomy-modeling strategy, model-based non-deep-learning object recog-nition, and deep learning object detection networks to achieve expert human-like performance. Methods: The AAR-DL approach consists of 4 key modules wherein prior knowledge (NI) is made use of judiciously at every stage. In the first module AAR-R, objects are recognized based on a previously created fuzzy anatomy model of the body region with all its organs following the automatic anatomy recognition (AAR) approach wherein high-level human anatomic knowledge is precisely codified. This module is purely model-based with no DL involvement. Although the AAR-R operation lacks accuracy, it is robust to artifacts and deviations (much like NI), and provides the much-needed anatomic guidance in the form of rough regions-of-interest (ROIs) for the following DL modules. The 2nd module DL-R makes use of the ROI information to limit the search region to just where each object is most likely to reside and performs DL-based detection of the 2D bounding boxes (BBs) in slices. The 2D BBs hug the shape of the 3D object much better than 3D BBs and their detection is feasible only due to anatomy guidance from AAR-R. In the 3rd module, the AAR model is deformed via the found 2D BBs providing refined model information which now embodies both NI and AI decisions. The refined AAR model more actively guides the 4th refined DL-R module to perform final object detection via DL. Anatomy knowledge is made use of in designing the DL networks wherein spatially sparse objects and non-sparse objects are handled differently to provide the required level of attention for each. Results: Utilizing 150 thoracic and 225 head and neck (H&N) computed tomography (CT) data sets of cancer patients undergoing routine radiation therapy planning, the recognition performance of the AAR-DL approach is evaluated on 10 thoracic and 16 H&N organs in comparison to pure model-based approach (AAR-R) and pure DL approach without anatomy guidance. Recognition accuracy is assessed via location error/ centroid distance error, scale or size error, and wall distance error. The results demonstrate how the errors are gradually and systematically reduced from the 1st module to the 4th module as high-level knowledge is infused via NI at various stages into the processing pipeline. This improvement is especially dramatic for sparse and artifact-prone challenging objects, achieving a location error over all objects of 4.4 mm and 4.3 mm for the two body regions, respectively. The pure DL approach failed on several very challenging sparse objects while AAR-DL achieved accurate recognition, almost matching human performance, showing the importance of anatomy guidance for robust operation. Anatomy guidance also reduces the time required for training DL networks considerably. Conclusions: (i) High-level anatomy guidance improves recognition performance of DL methods. (ii) This improvement is especially noteworthy for spatially sparse, low-contrast, inconspicuous, and artifact-prone objects. (iii) Once anatomy guidance is provided, 3D objects can be detected much more accurately via 2D BBs than 3D BBs and the 2D BBs represent object containment with much more specificity. (iv) Anatomy guidance brings stability and robustness to DL approaches for object localization. (v) The training time can be greatly reduced by making use of anatomy guidance. (c) 2022 Elsevier B.V. All rights reserved.

A Boosting Regression Approach to Medical Anatomy Detection

Anatomical landmark detection using multiple instance boosting with spatial regularization

Anatomic-landmark Detection Using Graphical Context Modelling.

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

Structured adaptive boosting trees for detection of multicellular aggregates in fluorescence intravital microscopy

Shape regression machine.

Shape regression machine and efficient segmentation of left ventricle endocardium from 2D B-mode echocardiogram

Image based regression using boosting method

Discriminative anatomy detection: Classification vs regression.

A Cascade Regression Model For Anatomical Landmark Detection

Robust Multi-scale Anatomical Landmark Detection in Incomplete 3D-CT Data

Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks

A Steering Engine: Learning 3-D Anatomy Orientation Using Regression Forests.

Object Recognition in Medical Images Via Anatomy-Guided Deep Learning.

An Adaptive Query Approach for Extracting Medical Images for Disease Detection Applications

Boosting the rule-out accuracy of deep disease detection using class weight modifiers

AEDAMIDL: An Enhanced and Discriminant Analysis of Medical Images using Deep Learning

Anatomy Detection and Localization in 3D Medical Images

Enhancing Multi-Object Detection in Ultrasound Images Through Semi-Supervised Learning, Focal Loss and Relation of Frame

Optimizing Object Detection Algorithms for Congenital Heart Diseases in Echocardiography: Exploring Bounding Box Sizes and Data Augmentation Techniques