Abstract:Background: Breast cancer is one of the most important malignant tumors among women causing a serious impact on women's lives and mammography is one the most important methods for breast examination. When diagnosing the breast disease, radiologists sometimes may consult some previous diagnosis cases as a reference. But there are many previous cases and it is important to find which cases are the similar cases, which is a big project costing lots of time. Medical image retrieval can provide objective reference information for doctors to diagnose disease. The method of fusing deep features can improve the retrieval accuracy, which solves the "semantic gap" problem caused by only using content features and location features. Methods: A similarity measure method combining deep feature for mammogram retrieval is proposed in this paper. First, the images are pre-processed to extract the low-level features, including content features and location features. Before extracting location features, registration with the standard image is performed. Then, the Convolutional Neural Network, the Stacked Auto-encoder Network, and the Deep Belief Network are built to extract the deep features, which are regarded as high-level features. Next, content similarity and deep similarity are calculated separately using the Euclidean distance between the query image and the dataset images. The location similarity is obtained by calculating the ratio of intersection to union of the mass regions. Finally, content similarity, location similarity, and deep similarity are fused to form the image fusion similarity. According to the similarity, the specified number of the most similar images can be returned. Results: In the experiment, 740 MLO mammograms are used, which are from women in Northeast China. The content similarity, location similarity, and deep similarity are fused by different weight coefficients. When only considering low-level features, the results are better with fusing 60% content feature similarity and 40% lesion location feature similarity. On this basis, CNN deep similarity, DBN deep similarity, and SAE deep similarity are fused separately. The experiments show that when fusing 60% DBN deep feature similarity and 40% low-level feature similarity, the results have obvious advantages. At this time, the precision is 0.745, recall is 0.850, comprehensive evaluation index is 0.794. Conclusions: We propose a similarity measure method fusing deep feature, content feature, and location feature. The retrieval results show that the precision and recall of this method have obvious advantage, compared with the content-based image retrieval and location-based image retrieval.

A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets

Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature

Histo-CADx: duo cascaded fusion stages for breast cancer diagnosis from histopathological images

Computer assisted recognition of breast cancer in biopsy images via fusion of nucleus-guided deep convolutional features

Feature Fusion and Ensemble Learning-Based CNN Model for Mammographic Image Classification

Benign and Malignant Breast Tumor Classification in Ultrasound and Mammography Images via Fusion of Deep Learning and Handcraft Features

A similarity measure method fusing deep feature for mammogram retrieval

Integrating intratumoral and peritumoral radiomics with deep transfer learning for DCE-MRI breast lesion differentiation: A multicenter study comparing performance with radiologists

A comprehensive investigation of multimodal deep learning fusion strategies for breast cancer classification

A Hybrid Deep Learning Approach to Predict Malignancy of Breast Lesions Using Mammograms

A novel fusion framework of deep bottleneck residual convolutional neural network for breast cancer classification from mammogram images

Classification of multi‐feature fusion ultrasound images of breast tumor within category 4 using convolutional neural networks

Breast Cancer Classification from Ultrasound Images Using Probability-Based Optimal Deep Learning Feature Fusion

Breast cancer diagnosis from contrast-enhanced mammography using multi-feature fusion neural network

Study on the classification of benign and malignant breast lesions using a multi-sequence breast MRI fusion radiomics and deep learning model

Deep learning in breast cancer risk assessment: evaluation of convolutional neural networks on a clinical dataset of full-field digital mammograms

Breast Cancer Detection Using Extreme Learning Machine Based on Feature Fusion with CNN Deep Features

Fusing Medical Image Features and Clinical Features with Deep Learning for Computer-Aided Diagnosis

Multimodality Computer-Aided Breast Cancer Diagnosis with FFDM and DCE-MRI

Digital mammographic tumor classification using transfer learning from deep convolutional neural networks

Deep Fusion of Human-Machine Knowledge with Attention Mechanism for Breast Cancer Diagnosis