Rearrangement of diflunisal acyl glucuronide into its beta-glucuronidase-resistant isomers facilitates transport through the small intestine to the colon of the rat.

R. Dickinson,A. R. King

DOI: https://doi.org/10.1016/S0024-3205(01)01371-6

2001-11-01

Life Science

Abstract:

What problem does this paper attempt to address?

Liver Histopathological Image Retrieval Based on Deep Metric Learning.

Pengshuai Yang,Yupeng Zhai,Lin Li,Hairong Lv,Jigang Wang,Chengzhan Zhu,Rui Jiang

DOI: https://doi.org/10.1109/bibm47256.2019.8982965

2019-01-01

Abstract:Histopathological image retrieval aims to search histopathological images sharing similar content with the query image, which could provide pathologists with an approach to easily obtain similar diagnostic cases for reference. Recent histopathological image retrieval methods are usually based on CNN feature extractors, which require a large amount of annotated data for training. Besides, most of existing methods could not define a reasonable similarity metric for histopathological images. In this paper, we apply deep metric learning to liver histopathological image retrieval. We construct a model based on mixed attention mechanism and train the model with a modified version of multi-similarity loss, which enables embedding vectors of similar images in the given metric space to be closer and dissimilar ones to be far from each other. Additionally, our model can be well fitted with limited data. Finally, we evaluate the proposed method with our own established liver histopathological image dataset. Compared with several published methods, our model shows higher performance.
Case-based similar image retrieval for weakly annotated large histopathological images of malignant lymphoma using deep metric learning

Noriaki Hashimoto,Yusuke Takagi,Hiroki Masuda,Hiroaki Miyoshi,Kei Kohno,Miharu Nagaishi,Kensaku Sato,Mai Takeuchi,Takuya Furuta,Keisuke Kawamoto,Kyohei Yamada,Mayuko Moritsubo,Kanako Inoue,Yasumasa Shimasaki,Yusuke Ogura,Teppei Imamoto,Tatsuzo Mishina,Ken Tanaka,Yoshino Kawaguchi,Shigeo Nakamura,Koichi Ohshima,Hidekata Hontani,Ichiro Takeuchi

DOI: https://doi.org/10.1016/j.media.2023.102752

IF: 10.9

2023-01-27

Medical Image Analysis

Abstract:In the present study, we propose a novel case-based similar image retrieval (SIR) method for hematoxylin and eosin (H&E) stained histopathological images of malignant lymphoma. When a whole slide image (WSI) is used as an input query, it is desirable to be able to retrieve similar cases by focusing on image patches in pathologically important regions such as tumor cells. To address this problem, we employ attention-based multiple instance learning, which enables us to focus on tumor-specific regions when the similarity between cases is computed. Moreover, we employ contrastive distance metric learning to incorporate immunohistochemical (IHC) staining patterns as useful supervised information for defining appropriate similarity between heterogeneous malignant lymphoma cases. In the experiment with 249 malignant lymphoma patients, we confirmed that the proposed method exhibited higher evaluation measures than the baseline case-based SIR methods. Furthermore, the subjective evaluation by pathologists revealed that our similarity measure using IHC staining patterns is appropriate for representing the similarity of H&E stained tissue images for malignant lymphoma.

engineering, biomedical,computer science, interdisciplinary applications, artificial intelligence,radiology, nuclear medicine & medical imaging
A Histopathological Image Feature Representation Method Based on Deep Learning

Gang Zhang,Ling Zhong,Yonghui Huang,Yi Zhang

DOI: https://doi.org/10.1109/itme.2015.34

2015-01-01

Abstract:Automated annotation and grading for histopathological image plays an important role in CAD systems. It provides valuable information and support for medical diagnosis. Currently, computer-aid analysis of histopathological images mainly relies on some well-designed digital features, which requires abundant human efforts and experiences in problem domain. Learning a good feature representation from data can have positive effects on constructing the target model. We propose a novel method for histopathological image feature representation based on deep learning. The method extracts high level representation of raw pixels of a local region through a network model with several hidden layers, which can learn potential features automatically. The proposed method is evaluated on a real data set from a large local hospital with comparison to two current state-of-the-art methods. The result is promising indicating that it achieves significant improvement of the model performance. Moreover, our study suggests that features learned through deep models can achieve better performance than human designed features.
A Similarity Measure of Histopathology Images by Deep Embeddings

Mehdi Afshari,H.R. Tizhoosh

DOI: https://doi.org/10.48550/arXiv.2107.13703

2021-07-29

Abstract:Histopathology digital scans are large-size images that contain valuable information at the pixel level. Content-based comparison of these images is a challenging task. This study proposes a content-based similarity measure for high-resolution gigapixel histopathology images. The proposed similarity measure is an expansion of cosine vector similarity to a matrix. Each image is divided into same-size patches with a meaningful amount of information (i.e., contained enough tissue). The similarity is measured by the extraction of patch-level deep embeddings of the last pooling layer of a pre-trained deep model at four different magnification levels, namely, 1x, 2.5x, 5x, and 10x magnifications. In addition, for faster measurement, embedding reduction is investigated. Finally, to assess the proposed method, an image search method is implemented. Results show that the similarity measure represents the slide labels with a maximum accuracy of 93.18\% for top-5 search at 5x magnification.

Image and Video Processing,Computer Vision and Pattern Recognition
Deep metric learning for image retrieval in smart city development

Qi Liu,Wenhan Li,Zhiyuan Chen,Bin Hua

DOI: https://doi.org/10.1016/j.scs.2021.103067

IF: 11.7

2021-10-01

Sustainable Cities and Society

Abstract:Deep metric learning (DML) aims to learn a consistent distance embedding where an anchor is closer within the same category than others. It underpins a variety of essential and significant tasks in the development of smart city including face recognition, landmark retrieval, pedestrian detection, person/vehicle re-identification, and so on. Traditional pair-based DML methods try to make full use of the data-to-data relations within a (mini-)batch, but they cannot grasp the data distribution information due to the batch size limitation. On the other hand, proxy-based DML schemes use different proxies to approximate the data distribution. However, the proxies are too sample to represent the intra-category variance. In this paper, we propose a simple but effective method, named soft-instance-label proxy, for embedding learning. It can capture the globe data distribution information while depicting the detailed intra-class data structure. The state-of-the-art empirical results on three public image retrieval benchmarks and two backbone networks demonstrate the superiority of our proposed method. Our Soft-instance-label proxy method can have a Recall<math>@1</math> improvement of 2.4 <math>%</math> with Googlenet, largely surpassing the current state-of-art-methods while demonstrating great potential in the development of smart city.

energy & fuels,construction & building technology,green & sustainable science & technology
Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology

Jiansheng Fang,Ming Zeng,Xiaoqing Zhang,Hongbo Liu,Yitian Zhao,Peng Zhang,Hong Yang,Junling Liu,Hanpei Miao,Yan Hu,Jiang Liu

DOI: https://doi.org/10.1016/j.bspc.2022.104277

IF: 5.1

2023-02-01

Biomedical Signal Processing and Control

Abstract:Fundus image retrieval can help ophthalmologists make evidence-based medico-decision by providing similar cases. Its basic task is to learn highly discriminative visual descriptors from image space, in which lesion features are the main differentiating clue. Lesions in fundus images appear small in size, similar in textures, and scatter around vessels, such as microaneurysms and hemorrhages. Hence, although a single small lesion has a saliently visual manifestation, its discriminative information is hard to reserve in the last image descriptors. For fundus images, the optic disc of the left and right eyes are symmetric, and the macular area lies in the central axis from the vertical view. Based on such spatial structure and lesion characteristics, we present a novel deep metric learning framework equipped with mirror attention to enhance the discriminative features of small and scattering lesions and encode them into image descriptors. The mirror attention can give lesions high attention scores by capturing spatial dependency of vertical and horizontal views, especially the relations between lesions and vessels. Based on the mirror attention, we further propose a new fine triplet loss to confine distances of positive pairs by exploiting the learned relevant degrees of positive pairs in a self-supervised manner. The fine triplet loss can help detect the subtle differences of positive pairs to improve the ranking performance of hit items. To demonstrate the effectiveness of improving retrieval performance, we conduct comprehensive experiments on the largest fundus dataset of diabetic retinopathy (DR) detection and achieve the best precision compared to counterparts. The experiments show that our method produces significant performance improvements for fundus image retrieval, especially the ranking quality of DR grades containing microaneurysms and hemorrhages. Our proposed mirror attention can be applied to off-the-shelf backbones and trained efficiently in an end-to-end manner for other medical images to obtain highly discriminative image descriptors.

engineering, biomedical
A similarity measure method fusing deep feature for mammogram retrieval

Zhiqiong Wang,Junchang Xin,Yukun Huang,Ling Xu,Jie Ren,Hao Zhang,Wei Qian,Xia Zhang,Jiren Liu

DOI: https://doi.org/10.3233/XST-190575

Abstract:Background: Breast cancer is one of the most important malignant tumors among women causing a serious impact on women's lives and mammography is one the most important methods for breast examination. When diagnosing the breast disease, radiologists sometimes may consult some previous diagnosis cases as a reference. But there are many previous cases and it is important to find which cases are the similar cases, which is a big project costing lots of time. Medical image retrieval can provide objective reference information for doctors to diagnose disease. The method of fusing deep features can improve the retrieval accuracy, which solves the "semantic gap" problem caused by only using content features and location features. Methods: A similarity measure method combining deep feature for mammogram retrieval is proposed in this paper. First, the images are pre-processed to extract the low-level features, including content features and location features. Before extracting location features, registration with the standard image is performed. Then, the Convolutional Neural Network, the Stacked Auto-encoder Network, and the Deep Belief Network are built to extract the deep features, which are regarded as high-level features. Next, content similarity and deep similarity are calculated separately using the Euclidean distance between the query image and the dataset images. The location similarity is obtained by calculating the ratio of intersection to union of the mass regions. Finally, content similarity, location similarity, and deep similarity are fused to form the image fusion similarity. According to the similarity, the specified number of the most similar images can be returned. Results: In the experiment, 740 MLO mammograms are used, which are from women in Northeast China. The content similarity, location similarity, and deep similarity are fused by different weight coefficients. When only considering low-level features, the results are better with fusing 60% content feature similarity and 40% lesion location feature similarity. On this basis, CNN deep similarity, DBN deep similarity, and SAE deep similarity are fused separately. The experiments show that when fusing 60% DBN deep feature similarity and 40% low-level feature similarity, the results have obvious advantages. At this time, the precision is 0.745, recall is 0.850, comprehensive evaluation index is 0.794. Conclusions: We propose a similarity measure method fusing deep feature, content feature, and location feature. The retrieval results show that the precision and recall of this method have obvious advantage, compared with the content-based image retrieval and location-based image retrieval.
A deep multi-branch attention model for histopathological breast cancer image classification

Rui Ding,Xiaoping Zhou,Dayu Tan,Yansen Su,Chao Jiang,Guo Yu,Chunhou Zheng

DOI: https://doi.org/10.1007/s40747-024-01398-z

IF: 6.7

2024-03-23

Complex & Intelligent Systems

Abstract:Abstract Since the impressive superior performance demonstrated by deep learning methods is widely used in histopathological image analysis and diagnosis, existing work cannot fully extract the information in the breast cancer images due to the limited high resolution of histopathological images. In this study, we construct a novel intermediate layer structure that fully extracts feature information and name it DMBANet, which can extract as much feature information as possible from the input image by up-dimensioning the intermediate convolutional layers to improve the performance of the network. Furthermore, we employ the depth-separable convolution method on the Spindle Structure by decoupling the intermediate convolutional layers and convolving them separately, to significantly reduce the number of parameters and computation of the Spindle Structure and improve the overall network operation speed. We also design the Spindle Structure as a multi-branch model and add different attention mechanisms to different branches. Spindle Structure can effectively improve the performance of the network, the branches with added attention can extract richer and more focused feature information, and the branch with residual connections can minimize the degradation phenomenon in our network and speed up network optimization. The comprehensive experiment shows the superior performance of DMBANet compared to the state-of-the-art method, achieving about 98% classification accuracy, which is better than existing methods. The code is available at https://github.com/Nagi-Dr/DMBANet-main .

computer science, artificial intelligence
Dual-branch hybrid encoding embedded network for histopathology image classification.

Mingshuai Li,Zhiqiu Hu,Song Qiu,Chenhao Zhou,Jialei Weng,Qiongzhu Dong,Xia Sheng,Ning Ren,Mei Zhou

DOI: https://doi.org/10.1088/1361-6560/acf556

IF: 3.5

2023-01-01

Physics in Medicine and Biology

Abstract:Learning-based histopathology image (HI) classification methods serve as important tools for auxiliary diagnosis in the prognosis stage. However, most existing methods are focus on a single target cancer due to inter-domain differences among different cancer types, limiting their applicability to different cancer types. To overcome these limitations, this paper presents a high-performance HI classification method that aims to address inter-domain differences and provide an improved solution for reliable and practical HI classification.Approach.Firstly, we collect a high-quality hepatocellular carcinoma (HCC) dataset with enough data to verify the stability and practicability of the method. Secondly, a novel dual-branch hybrid encoding embedded network is proposed, which integrates the feature extraction capabilities of convolutional neural network and Transformer. This well-designed structure enables the network to extract diverse features while minimizing redundancy from a single complex network. Lastly, we develop a salient area constraint loss function tailored to the unique characteristics of HIs to address inter-domain differences and enhance the robustness and universality of the methods.Main results.Extensive experiments have conducted on the proposed HCC dataset and two other publicly available datasets. The proposed method demonstrates outstanding performance with an impressive accuracy of 99.09\% on the HCC dataset and achieves state-of-the-art results on the other two public datasets. These remarkable outcomes underscore the superior performance and versatility of our approach in multiple HI classification.Significance.The advancements presented in this study contribute to the field of HI analysis by providing a reliable and practical solution for multiple cancer classification, potentially improving diagnostic accuracy and patient outcomes. Our code is available at https://github.com/lms-design/DHEE-net.&#xD.
Classification in Histopathology: A unique deep embeddings extractor for multiple classification tasks

Adrien Nivaggioli,Nicolas Pozin,Rémy Peyret,Stéphane Sockeel,Marie Sockeel,Nicolas Nerrienet,Marceau Clavel,Clara Simmat,Catherine Miquel

DOI: https://doi.org/10.48550/arXiv.2303.05180

2023-03-09

Abstract:In biomedical imaging, deep learning-based methods are state-of-the-art for every modality (virtual slides, MRI, etc.) In histopathology, these methods can be used to detect certain biomarkers or classify lesions. However, such techniques require large amounts of data to train high-performing models which can be intrinsically difficult to acquire, especially when it comes to scarce biomarkers. To address this challenge, we use a single, pre-trained, deep embeddings extractor to convert images into deep features and train small, dedicated classification head on these embeddings for each classification task. This approach offers several benefits such as the ability to reuse a single pre-trained deep network for various tasks; reducing the amount of labeled data needed as classification heads have fewer parameters; and accelerating training time by up to 1000 times, which allows for much more tuning of the classification head. In this work, we perform an extensive comparison of various open-source backbones and assess their fit to the target histological image domain. This is achieved using a novel method based on a proxy classification task. We demonstrate that thanks to this selection method, an optimal feature extractor can be selected for different tasks on the target domain. We also introduce a feature space augmentation strategy which proves to substantially improve the final metrics computed for the different tasks considered. To demonstrate the benefit of such backbone selection and feature-space augmentation, our experiments are carried out on three separate classification tasks and show a clear improvement on each of them: microcalcifications (29.1% F1-score increase), lymph nodes metastasis (12.5% F1-score increase), mitosis (15.0% F1-score increase).

Computer Vision and Pattern Recognition,Artificial Intelligence
Look, Investigate, and Classify: A Deep Hybrid Attention Method for Breast Cancer Classification

Bolei Xu,Jingxin Liu,Xianxu Hou,Bozhi Liu,Jon Garibaldi,Ian O. Ellis,Andy Green,Linlin Shen,Guoping Qiu

DOI: https://doi.org/10.48550/arXiv.1902.10946

2019-02-28

Abstract:One issue with computer based histopathology image analysis is that the size of the raw image is usually very large. Taking the raw image as input to the deep learning model would be computationally expensive while resizing the raw image to low resolution would incur information loss. In this paper, we present a novel deep hybrid attention approach to breast cancer classification. It first adaptively selects a sequence of coarse regions from the raw image by a hard visual attention algorithm, and then for each such region it is able to investigate the abnormal parts based on a soft-attention mechanism. A recurrent network is then built to make decisions to classify the image region and also to predict the location of the image region to be investigated at the next time step. As the region selection process is non-differentiable, we optimize the whole network through a reinforcement approach to learn an optimal policy to classify the regions. Based on this novel Look, Investigate and Classify approach, we only need to process a fraction of the pixels in the raw image resulting in significant saving in computational resources without sacrificing performances. Our approach is evaluated on a public breast cancer histopathology database, where it demonstrates superior performance to the state-of-the-art deep learning approaches, achieving around 96\% classification accuracy while only 15% of raw pixels are used.

Computer Vision and Pattern Recognition
A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images

Chulong Zhang,Yuming Jiang,Na Li,Zhicheng Zhang,Md Tauhidul Islam,Jingjing Dai,Lin Liu,Wenfeng He,Wenjian Qin,Jing Xiong,Yaoqin Xie,Xiaokun Liang

DOI: https://doi.org/10.48550/arXiv.2208.07655

2023-04-10

Abstract:Pathologists need to combine information from differently stained pathology slices for accurate diagnosis. Deformable image registration is a necessary technique for fusing multi-modal pathology slices. This paper proposes a hybrid deep feature-based deformable image registration framework for stained pathology samples. We first extract dense feature points via the detector-based and detector-free deep learning feature networks and perform points matching. Then, to further reduce false matches, an outlier detection method combining the isolation forest statistical model and the local affine correction model is proposed. Finally, the interpolation method generates the deformable vector field for pathology image registration based on the above matching points. We evaluate our method on the dataset of the Non-rigid Histology Image Registration (ANHIR) challenge, which is co-organized with the IEEE ISBI 2019 conference. Our technique outperforms the traditional approaches by 17% with the Average-Average registration target error (rTRE) reaching 0.0034. The proposed method achieved state-of-the-art performance and ranked 1st in evaluating the test dataset. The proposed hybrid deep feature-based registration method can potentially become a reliable method for pathology image registration.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning,Medical Physics
LEARNING BINARY SEMANTIC EMBEDDING FOR BREAST HISTOLOGY IMAGE CLASSIFICATION AND RETRIEVAL

Xiao Kang,Xingbo Liu,Xiushan Nie,Yilong Yin

DOI: https://doi.org/10.1109/icassp39728.2021.9415036

2021-01-01

Abstract:With the development of medical imaging technology and machine learning, the computer-assisted diagnosis has attracted extensive research attention, which can provide beneficial reference to pathologists. However, the exponential growth of medical images and uninterpretability of traditional classification models have hindered the applications of the computer-assisted diagnosis. To address this issues, we propose a novel method for Learning Binary Semantic Embedding (LBSE). Based on this efficient and effective embedding, classification and retrieval are performed to provide interpretable computer-assisted diagnosis for histology images. Furthermore, double supervision, bit uncorrelation and balance constraint, asymmetric strategy and discrete optimization are seamlessly integrated in the proposed method for learning binary embedding. Experiments conducted on three benchmark datasets validate the superiority of LBSE under various scenarios.
Enhancing image retrieval through entropy-based deep metric learning

Kambiz Rahbar,Fatemeh Taheri

DOI: https://doi.org/10.1007/s11042-024-19296-4

IF: 2.577

2024-05-08

Multimedia Tools and Applications

Abstract:The increasing demand for effective retrieval from image datasets has been driven by the rapid growth of digital images. Image retrieval is a method for creating a structured database based on searching for similar images to a user's query image. Effective image representation and extraction of distinctive features are among the challenges faced by image retrieval systems. The proposed approach utilizes a triplet loss function based on binary cross-entropy to train a Siamese network, allowing for deep metric learning and creating a discriminative feature space with maximum discrimination between classes and minimum intra-class distance. In this approach, a pre-trained neural network is serialized with Siamese network. Initially, image features are extracted using a pre-trained convolutional neural network. Then, a Siamese network is trained to create a discriminative feature space using deep metric learning. Learning in this method is controlled by a triplet loss function based on binary cross-entropy with anchor, positive, and negative samples. The transformation from the initial feature space to the discriminative feature space is achieved through deep metric learning in the Siamese network. The proposed approach proves to effective in discriminating features of diverse classes in the dataset. Visualization of the feature space is also demonstrated using the t-SNE statistical technique. Furthermore, the explainability of the proposed approach is presented by examining the Shapley value. The proposed approach was examined on the Corel10K, Caltech101, and Caltech256 datasets. The best reported results with the precision metric were 0.988 for the Corel10K dataset and 0.951 and 0.902 for the Caltech101 and Caltech256 datasets, respectively. Experimental results show that the proposed approach effectively improves the retrieval results of similar samples to the query image creating discrimination in the feature vectors, even with dimensionality reduction.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
Deep metric learning-based image retrieval system for chest radiograph and its clinical applications in COVID-19

Aoxiao Zhong,Xiang Li,Dufan Wu,Hui Ren,Kyungsang Kim,Younggon Kim,Varun Buch,Nir Neumark,Bernardo Bizzo,Won Young Tak,Soo Young Park,Yu Rim Lee,Min Kyu Kang,Jung Gil Park,Byung Seok Kim,Woo Jin Chung,Ning Guo,Ittai Dayan,Mannudeep K Kalra,Quanzheng Li,Mannudeep K. Kalra

DOI: https://doi.org/10.1016/j.media.2021.101993

IF: 10.9

2021-05-01

Medical Image Analysis

Abstract:In recent years, deep learning-based image analysis methods have been widely applied in computer-aided detection, diagnosis and prognosis, and has shown its value during the public health crisis of the novel coronavirus disease 2019 (COVID-19) pandemic. Chest radiograph (CXR) has been playing a crucial role in COVID-19 patient triaging, diagnosing and monitoring, particularly in the United States. Considering the mixed and unspecific signals in CXR, an image retrieval model of CXR that provides both similar images and associated clinical information can be more clinically meaningful than a direct image diagnostic model. In this work we develop a novel CXR image retrieval model based on deep metric learning. Unlike traditional diagnostic models which aim at learning the direct mapping from images to labels, the proposed model aims at learning the optimized embedding space of images, where images with the same labels and similar contents are pulled together. The proposed model utilizes multi-similarity loss with hard-mining sampling strategy and attention mechanism to learn the optimized embedding space, and provides similar images, the visualizations of disease-related attention maps and useful clinical information to assist clinical decisions. The model is trained and validated on an international multi-site COVID-19 dataset collected from 3 different sources. Experimental results of COVID-19 image retrieval and diagnosis tasks show that the proposed model can serve as a robust solution for CXR analysis and patient management for COVID-19. The model is also tested on its transferability on a different clinical decision support task for COVID-19, where the pre-trained model is applied to extract image features from a new dataset without any further training. The extracted features are then combined with COVID-19 patient's vitals, lab tests and medical histories to predict the possibility of airway intubation in 72 hours, which is strongly associated with patient prognosis, and is crucial for patient care and hospital resource planning. These results demonstrate our deep metric learning based image retrieval model is highly efficient in the CXR retrieval, diagnosis and prognosis, and thus has great clinical value for the treatment and management of COVID-19 patients.

engineering, biomedical,computer science, interdisciplinary applications, artificial intelligence,radiology, nuclear medicine & medical imaging
Histopathological image classification with deep convolutional neural networks

Zahangir Alom,Theus Aspiras,Tarek M. Taha,Vijayan K. Asari,Tarek Taha

DOI: https://doi.org/10.1117/12.2530291

2019-09-06

Abstract:In the last few years, deep learning approaches have been applied successfully in different modalities of medical imaging problems and achieved state-of-the-art accuracy. Due to the huge volume and variety of imaging modalities, it remains a large open research area. However, in this paper, we have applied Inception Residual Recurrent Convolutional Neural Network (IRRCNN) model for histopathological image classification where a new publicly available dataset named KIMIA Path960 is used. This database contains 960 histopathological images with 20 different classes (different types of tissue collected from 400 Whole Slide Images). In this implementation, we have evaluated the model with non-overlapping patches size of 64×64 pixels and the variant of samples are generated from each patch with different data augmentation techniques including rotation, shear, zooming, and horizontal and vertical flipping. The experimental results are compared against Linear Binary Pattern (LBP), bag-of-visual words (BoVW), and deep learning method with AlexNet and VGG16 networks. The IRRCNN model shows around 98.79% testing accuracy for augmented patch-level evaluation which is around 2.29% and 4% superior performance compared to Support Vector Machine with histogram intersection kernel (IKSVM) with BoVW and VGG16 methods respectively. Additionally, this evaluation also demonstrates that the deep feature representation-based method outperforms compared to a traditional feature-based method including LBP and BoVW for the histopathological image classification problem.
Histopathological Image Classification with Cell Morphology Aware Deep Neural Networks

Andrey Ignatov,Josephine Yates,Valentina Boeva

2024-07-12

Abstract:Histopathological images are widely used for the analysis of diseased (tumor) tissues and patient treatment selection. While the majority of microscopy image processing was previously done manually by pathologists, recent advances in computer vision allow for accurate recognition of lesion regions with deep learning-based solutions. Such models, however, usually require extensive annotated datasets for training, which is often not the case in the considered task, where the number of available patient data samples is very limited. To deal with this problem, we propose a novel DeepCMorph model pre-trained to learn cell morphology and identify a large number of different cancer types. The model consists of two modules: the first one performs cell nuclei segmentation and annotates each cell type, and is trained on a combination of 8 publicly available datasets to ensure its high generalizability and robustness. The second module combines the obtained segmentation map with the original microscopy image and is trained for the downstream task. We pre-trained this module on the Pan-Cancer TCGA dataset consisting of over 270K tissue patches extracted from 8736 diagnostic slides from 7175 patients. The proposed solution achieved a new state-of-the-art performance on the dataset under consideration, detecting 32 cancer types with over 82% accuracy and outperforming all previously proposed solutions by more than 4%. We demonstrate that the resulting pre-trained model can be easily fine-tuned on smaller microscopy datasets, yielding superior results compared to the current top solutions and models initialized with ImageNet weights. The codes and pre-trained models presented in this paper are available at: <a class="link-external link-https" href="https://github.com/aiff22/DeepCMorph" rel="external noopener nofollow">this https URL</a>

Image and Video Processing,Computer Vision and Pattern Recognition,Quantitative Methods
Ranking Loss and Sequestering Learning for Reducing Image Search Bias in Histopathology

Pooria Mazaheri,Azam Asilian Bidgoli,Shahryar Rahnamayan,H.R. Tizhoosh

2023-04-15

Abstract:Recently, deep learning has started to play an essential role in healthcare applications, including image search in digital pathology. Despite the recent progress in computer vision, significant issues remain for image searching in histopathology archives. A well-known problem is AI bias and lack of generalization. A more particular shortcoming of deep models is the ignorance toward search functionality. The former affects every model, the latter only search and matching. Due to the lack of ranking-based learning, researchers must train models based on the classification error and then use the resultant embedding for image search purposes. Moreover, deep models appear to be prone to internal bias even if using a large image repository of various hospitals. This paper proposes two novel ideas to improve image search performance. First, we use a ranking loss function to guide feature extraction toward the matching-oriented nature of the search. By forcing the model to learn the ranking of matched outputs, the representation learning is customized toward image search instead of learning a class label. Second, we introduce the concept of sequestering learning to enhance the generalization of feature extraction. By excluding the images of the input hospital from the matched outputs, i.e., sequestering the input domain, the institutional bias is reduced. The proposed ideas are implemented and validated through the largest public dataset of whole slide images. The experiments demonstrate superior results compare to the-state-of-art.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Learning how to detect: A deep reinforcement learning method for whole-slide melanoma histopathology images

Tingting Zheng,Weixing Chen,Shuqin Li,Hao Quan,Mingchen Zou,Song Zheng,Yue Zhao,Xinghua Gao,Xiaoyu Cui

DOI: https://doi.org/10.1016/j.compmedimag.2023.102275

IF: 7.422

2023-07-31

Computerized Medical Imaging and Graphics

Abstract:Cutaneous melanoma represents one of the most life-threatening malignancies. Histopathological image analysis serves as a vital tool for early melanoma detection. Deep neural network (DNN) models are frequently employed to aid pathologists in enhancing the efficiency and accuracy of diagnoses. However, due to the paucity of well-annotated, high-resolution, whole-slide histopathology image (WSI) datasets, WSIs are typically fragmented into numerous patches during the model training and testing stages. This process disregards the inherent interconnectedness among patches, potentially impeding the models' performance. Additionally, the presence of excess, non-contributing patches extends processing times and introduces substantial computational burdens. To mitigate these issues, we draw inspiration from the clinical decision-making processes of dermatopathologists to propose an innovative, weakly supervised deep reinforcement learning framework, titled Fast medical decision-making in melanoma histopathology images (FastMDP-RL). This framework expedites model inference by reducing the number of irrelevant patches identified within WSIs. FastMDP-RL integrates two DNN-based agents: the search agent (SeAgent) and the decision agent (DeAgent). The SeAgent initiates actions, steered by the image features observed in the current viewing field at various magnifications. Simultaneously, the DeAgent provides labeling probabilities for each patch. We utilize multi-instance learning (MIL) to construct a teacher-guided model (MILTG), serving a dual purpose: rewarding the SeAgent and guiding the DeAgent. Our evaluations were conducted using two melanoma datasets: the publicly accessible TCIA-CM dataset and the proprietary MELSC dataset. Our experimental findings affirm FastMDP-RL's ability to expedite inference and accurately predict WSIs, even in the absence of pixel-level annotations. Moreover, our research investigates the WSI-based interactive environment, encompassing the design of agents, state and reward functions, and feature extractors suitable for melanoma tissue images. This investigation offers valuable insights and references for researchers engaged in related studies. The code is available at: https://github.com/titizheng/FastMDP-RL .

engineering, biomedical,radiology, nuclear medicine & medical imaging
Semi-HIC: A novel semi-supervised deep learning method for histopathological image classification

Lei Su,Yu Liu,Minghui Wang,Ao Li

DOI: https://doi.org/10.1016/j.compbiomed.2021.104788

IF: 7.7

2021-10-01

Computers in Biology and Medicine

Abstract:Histopathological images provide a gold standard for cancer recognition and diagnosis. Existing approaches for histopathological image classification are supervised learning methods that demand a large amount of labeled data to obtain satisfying performance, which have to face the challenge of limited data annotation due to prohibitive time cost. To circumvent this shortage, a promising strategy is to design semi-supervised learning methods. Recently, a novel semi-supervised approach called Learning by Association (LA) is proposed, which achieves promising performance in nature image classification. However, there are still great challenges in its application to histopathological image classification due to the wide inter-class similarity and intra-class heterogeneity in histopathological images. To address these issues, we propose a novel semi-supervised deep learning method called Semi-HIC for histopathological image classification. Particularly, we introduce a new semi-supervised loss function combining an association cycle consistency (ACC) loss and a maximal conditional association (MCA) loss, which can take advantage of a large number of unlabeled patches and address the problems of inter-class similarity and intra-class variation in histopathological images, and thereby remarkably improve classification performance for histopathological images. Besides, we employ an efficient network architecture with cascaded Inception blocks (CIBs) to learn rich and discriminative embeddings from patches. Experimental results on both the Bioimaging 2015 challenge dataset and the BACH dataset demonstrate our Semi-HIC method compares favorably with existing deep learning methods for histopathological image classification and consistently outperforms the semi-supervised LA method.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology

Rearrangement of diflunisal acyl glucuronide into its beta-glucuronidase-resistant isomers facilitates transport through the small intestine to the colon of the rat.

Liver Histopathological Image Retrieval Based on Deep Metric Learning.

Case-based similar image retrieval for weakly annotated large histopathological images of malignant lymphoma using deep metric learning

A Histopathological Image Feature Representation Method Based on Deep Learning

A Similarity Measure of Histopathology Images by Deep Embeddings

Deep metric learning for image retrieval in smart city development

Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology

A similarity measure method fusing deep feature for mammogram retrieval

A deep multi-branch attention model for histopathological breast cancer image classification

Dual-branch hybrid encoding embedded network for histopathology image classification.

Classification in Histopathology: A unique deep embeddings extractor for multiple classification tasks

Look, Investigate, and Classify: A Deep Hybrid Attention Method for Breast Cancer Classification

A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathology Images

LEARNING BINARY SEMANTIC EMBEDDING FOR BREAST HISTOLOGY IMAGE CLASSIFICATION AND RETRIEVAL

Enhancing image retrieval through entropy-based deep metric learning

Deep metric learning-based image retrieval system for chest radiograph and its clinical applications in COVID-19

Histopathological image classification with deep convolutional neural networks

Histopathological Image Classification with Cell Morphology Aware Deep Neural Networks

Ranking Loss and Sequestering Learning for Reducing Image Search Bias in Histopathology

Learning how to detect: A deep reinforcement learning method for whole-slide melanoma histopathology images

Semi-HIC: A novel semi-supervised deep learning method for histopathological image classification