Explorer Atypical Femoral Fracture in Osteoporosis Pseudoglioma Syndrome Associated With Two Novel Compound Heterozygous Mutations in LRP 5

N. Alonso,D. Soares,E. McCloskey,G. Summers,S. Ralston,C. Gregson

Abstract:

What problem does this paper attempt to address?

IRLSG: Invariant Representation Learning for Single-Domain Generalization in Medical Image Segmentation

Ziwei Niu,Hao Sun,Shuyi Ouyang,Shiao Xie,Yen-Wei Chen,Ruofeng Tong,Lanfen Lin

DOI: https://doi.org/10.1109/icassp48485.2024.10446700

2024-01-01

Abstract:Single-domain generalization (SDG) can efficiently enhance model generalization while avoiding high annotation costs and privacy concerns. However, existing SDG methods are mainly based on data manipulation and meta-learning, which are not efficient enough due to the limited generalization performance and complex inference. In response to these challenges, we present a novel single domaininvariant representation learning approach for medical image segmentation, called IRLSG, with two appealing designs: (1) A Classscale Photo-metric Augmentation is first proposed to simulate unseen target domain that is sufficient in diversity and informativeness. After that, a Dual-Consistency Framework is further designed to constrain the consistency of intermediate features and segmentation results between the original and the augmented images, which helps to explore the domain-invariant representation. (2) A simple and effective Style Feature Whitening is designed to decouple and remove the domain-specific style from higher-order covariance statistics, which can further improve the modeling and generalization capability of the network. Experimental results on different benchmarks demonstrate that our IRLSG outperforms the current state-of-the-art methods in tackling single-domain generalization.
Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis

Xiaomeng Li,Xiaowei Hu,Xiaojuan Qi,Lequan Yu,Wei Zhao,Pheng-Ann Heng,Lei Xing

DOI: https://doi.org/10.1109/tmi.2021.3075244

IF: 10.6

2021-09-01

IEEE Transactions on Medical Imaging

Abstract:The automatic diagnosis of various conventional ophthalmic diseases from fundus images is important in clinical practice. However, developing such automatic solutions is challenging due to the requirement of a large amount of training data and the expensive annotations for medical images. This paper presents a novel self-supervised learning framework for retinal disease diagnosis to reduce the annotation efforts by learning the visual features from the unlabeled images. To achieve this, we present a rotation-oriented collaborative method that explores rotation-related and rotation-invariant features, which capture discriminative structures from fundus images and also explore the invariant property used for retinal disease classification. We evaluate the proposed method on two public benchmark datasets for retinal disease classification. The experimental results demonstrate that our method outperforms other self-supervised feature learning methods (around 4.2% area under the curve (AUC)). With a large amount of unlabeled data available, our method can surpass the supervised baseline for pathologic myopia (PM) and is very close to the supervised baseline for age-related macular degeneration (AMD), showing the potential benefit of our method in clinical practice.

engineering, biomedical,imaging science & photographic technology, electrical & electronic,computer science, interdisciplinary applications,radiology, nuclear medicine & medical imaging
Examining the quality of learned representations in self-supervised medical image analysis: a comprehensive review and empirical study

Kaliprasad Pani,Indu Chawla

DOI: https://doi.org/10.1007/s11042-024-19072-4

IF: 2.577

2024-04-19

Multimedia Tools and Applications

Abstract:Medical Image Analysis (MIA) is integral to healthcare, demanding advanced computational techniques for precise diagnostics and treatment planning. The demand for accurate and interpretable models is imperative in the ever-evolving healthcare landscape. This paper explores the potential of Self-Supervised Learning (SSL), transfer learning and domain adaptation methods in MIA. The study comprehensively reviews SSL-based computational techniques in the context of medical imaging, highlighting their merits and limitations. In an empirical investigation, this study examines the lack of interpretable and explainable component selection in existing SSL approaches for MIA. Unlike prior studies that randomly select SSL components based on their performance on natural images, this paper focuses on identifying components based on the quality of learned representations through various clustering evaluation metrics. Various SSL techniques and backbone combinations were rigorously assessed on diverse medical image datasets. The results of this experiment provided insights into the performance and behavior of SSL methods, paving the way for an explainable and interpretable component selection mechanism for artificial intelligence models in medical imaging. The empirical study reveals the superior performance of BYOL (Bootstrap Your Own Latent) with resnet as the backbone, as indicated by various clustering evaluation metrics such as Silhouette Coefficient (0.6), Davies-Bouldin Index (0.67), and Calinski-Harabasz Index (36.9). The study also emphasizes the benefits of transferring weights from a model trained on a similar dataset instead of a dataset from a different domain. Results indicate that the proposed mechanism expedited convergence, achieving 98.66% training accuracy and 92.48% testing accuracy in 23 epochs, requiring almost half the number of epochs for similar results with ImageNet weights. This research contributes to advancing the understanding of SSL in MIA, providing valuable insights for enhancing the reliability of artificial intelligence models in clinical applications.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
OPTiML: Dense Semantic Invariance Using Optimal Transport for Self-Supervised Medical Image Representation

Azad Singh,Vandan Gorade,Deepak Mishra

2024-05-12

Abstract:Self-supervised learning (SSL) has emerged as a promising technique for medical image analysis due to its ability to learn without annotations. However, despite the promising potential, conventional SSL methods encounter limitations, including challenges in achieving semantic alignment and capturing subtle details. This leads to suboptimal representations, which fail to accurately capture the underlying anatomical structures and pathological details. In response to these constraints, we introduce a novel SSL framework OPTiML, employing optimal transport (OT), to capture the dense semantic invariance and fine-grained details, thereby enhancing the overall effectiveness of SSL in medical image representation learning. The core idea is to integrate OT with a cross-viewpoint semantics infusion module (CV-SIM), which effectively captures complex, fine-grained details inherent in medical images across different viewpoints. In addition to the CV-SIM module, OPTiML imposes the variance and covariance regularizations within OT framework to force the model focus on clinically relevant information while discarding less informative features. Through these, the proposed framework demonstrates its capacity to learn semantically rich representations that can be applied to various medical imaging tasks. To validate its effectiveness, we conduct experimental studies on three publicly available datasets from chest X-ray modality. Our empirical results reveal OPTiML's superiority over state-of-the-art methods across all evaluated tasks.

Computer Vision and Pattern Recognition,Machine Learning
Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging

Shekoofeh Azizi,Laura Culp,Jan Freyberg,Basil Mustafa,Sebastien Baur,Simon Kornblith,Ting Chen,Nenad Tomasev,Jovana Mitrović,Patricia Strachan,S. Sara Mahdavi,Ellery Wulczyn,Boris Babenko,Megan Walker,Aaron Loh,Po-Hsuan Cameron Chen,Yuan Liu,Pinal Bavishi,Scott Mayer McKinney,Jim Winkens,Abhijit Guha Roy,Zach Beaver,Fiona Ryan,Justin Krogue,Mozziyar Etemadi,Umesh Telang,Yun Liu,Lily Peng,Greg S. Corrado,Dale R. Webster,David Fleet,Geoffrey Hinton,Neil Houlsby,Alan Karthikesalingam,Mohammad Norouzi,Vivek Natarajan

DOI: https://doi.org/10.1038/s41551-023-01049-7

IF: 28.1

2023-06-09

Nature Biomedical Engineering

Abstract:Machine-learning models for medical tasks can match or surpass the performance of clinical experts. However, in settings differing from those of the training dataset, the performance of a model can deteriorate substantially. Here we report a representation-learning strategy for machine-learning models applied to medical-imaging tasks that mitigates such 'out of distribution' performance problem and that improves model robustness and training efficiency. The strategy, which we named REMEDIS (for 'Robust and Efficient Medical Imaging with Self-supervision'), combines large-scale supervised transfer learning on natural images and intermediate contrastive self-supervised learning on medical images and requires minimal task-specific customization. We show the utility of REMEDIS in a range of diagnostic-imaging tasks covering six imaging domains and 15 test datasets, and by simulating three realistic out-of-distribution scenarios. REMEDIS improved in-distribution diagnostic accuracies up to 11.5% with respect to strong supervised baseline models, and in out-of-distribution settings required only 1–33% of the data for retraining to match the performance of supervised models retrained using all available data. REMEDIS may accelerate the development lifecycle of machine-learning models for medical imaging.

engineering, biomedical
A rotation meanout network with invariance for dermoscopy image classification and retrieval

Yilan Zhang,Fengying Xie,Xuedong Song,Hangning Zhou,Yiguang Yang,Haopeng Zhang,Jie Liu

DOI: https://doi.org/10.1016/j.compbiomed.2022.106272

IF: 7.7

2022-11-10

Computers in Biology and Medicine

Abstract:The computer-aided diagnosis (CAD) system can provide a reference basis for the clinical diagnosis of skin diseases. Convolutional neural networks (CNNs) can not only extract visual elements such as colors and shapes but also semantic features. As such they have made great improvements in many tasks of dermoscopy images. The imaging of dermoscopy has no principal orientation, indicating that there are a large number of skin lesion rotations in the datasets. However, CNNs lack rotation invariance, which is bound to affect the robustness of CNNs against rotations. To tackle this issue, we propose a rotation meanout (RM) network to extract rotation-invariant features from dermoscopy images. In RM, each set of rotated feature maps corresponds to a set of outputs of the weight-sharing convolutions and they are fused using meanout strategy to obtain the final feature maps. Through theoretical derivation, the proposed RM network is rotation-equivariant and can extract rotation-invariant features when followed by the global average pooling (GAP) operation. The extracted rotation-invariant features can better represent the original data in classification and retrieval tasks for dermoscopy images. The RM is a general operation, which does not change the network structure or increase any parameters, and can be flexibly embedded in any part of CNNs. Extensive experiments are conducted on a dermoscopy image dataset. The results show that our method outperforms other anti-rotation methods and achieves great improvements in skin disease classification and retrieval tasks, indicating the potential of rotation invariance in the field of dermoscopy images.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology
Unsupervised Representation Learning by Predicting Image Rotations

Spyros Gidaris,Praveer Singh,Nikos Komodakis

DOI: https://doi.org/10.48550/arXiv.1803.07728

2018-03-21

Computer Vision and Pattern Recognition

Abstract:Over the last years, deep convolutional neural networks (ConvNets) have transformed the field of computer vision thanks to their unparalleled capacity to learn high level semantic image features. However, in order to successfully learn those features, they usually require massive amounts of manually labeled data, which is both expensive and impractical to scale. Therefore, unsupervised semantic feature learning, i.e., learning without requiring manual annotation effort, is of crucial importance in order to successfully harvest the vast amount of visual data that are available today. In our work we propose to learn image features by training ConvNets to recognize the 2d rotation that is applied to the image that it gets as input. We demonstrate both qualitatively and quantitatively that this apparently simple task actually provides a very powerful supervisory signal for semantic feature learning. We exhaustively evaluate our method in various unsupervised feature learning benchmarks and we exhibit in all of them state-of-the-art performance. Specifically, our results on those benchmarks demonstrate dramatic improvements w.r.t. prior state-of-the-art approaches in unsupervised representation learning and thus significantly close the gap with supervised feature learning. For instance, in PASCAL VOC 2007 detection task our unsupervised pre-trained AlexNet model achieves the state-of-the-art (among unsupervised methods) mAP of 54.4% that is only 2.4 points lower from the supervised case. We get similarly striking results when we transfer our unsupervised learned features on various other tasks, such as ImageNet classification, PASCAL classification, PASCAL segmentation, and CIFAR-10 classification. The code and models of our paper will be published on: https://github.com/gidariss/FeatureLearningRotNet .
Self-Supervised Representation Learning by Rotation Feature Decoupling

Zeyu Feng,Chang Xu,Dacheng Tao

DOI: https://doi.org/10.1109/cvpr.2019.01061

2019-06-01

Abstract:We introduce a self-supervised learning method that focuses on beneficial properties of representation and their abilities in generalizing to real-world tasks. The method incorporates rotation invariance into the feature learning framework, one of many good and well-studied properties of visual representation, which is rarely appreciated or exploited by previous deep convolutional neural network based self-supervised representation learning methods. Specifically, our model learns a split representation that contains both rotation related and unrelated parts. We train neural networks by jointly predicting image rotations and discriminating individual instances. In particular, our model decouples the rotation discrimination from instance discrimination, which allows us to improve the rotation prediction by mitigating the influence of rotation label noise, as well as discriminate instances without regard to image rotations. The resulting feature has a better generalization ability for more various tasks. Experimental results show that our model outperforms current state-of-the-art methods on standard self-supervised feature learning benchmarks.
3D Self-Supervised Methods for Medical Imaging

Aiham Taleb,Winfried Loetzsch,Noel Danz,Julius Severin,Thomas Gaertner,Benjamin Bergner,Christoph Lippert

DOI: https://doi.org/10.48550/arXiv.2006.03829

2020-06-06

Computer Vision and Pattern Recognition

Abstract:Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for expert annotation. The developed algorithms are 3D Contrastive Predictive Coding, 3D Rotation prediction, 3D Jigsaw puzzles, Relative 3D patch location, and 3D Exemplar networks. Our experiments show that pretraining models with our 3D tasks yields more powerful semantic representations, and enables solving downstream tasks more accurately and efficiently, compared to training the models from scratch and to pretraining them on 2D slices. We demonstrate the effectiveness of our methods on three downstream tasks from the medical imaging domain: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT, and iii) Diabetic Retinopathy Detection from 2D Fundus images. In each task, we assess the gains in data-efficiency, performance, and speed of convergence. Interestingly, we also find gains when transferring the learned representations, by our methods, from a large unlabeled 3D corpus to a small downstream-specific dataset. We achieve results competitive to state-of-the-art solutions at a fraction of the computational expense. We publish our implementations for the developed algorithms (both 3D and 2D versions) as an open-source library, in an effort to allow other researchers to apply and extend our methods on their datasets.
Learning Self-Supervised Representations for Label Efficient Cross-Domain Knowledge Transfer on Diabetic Retinopathy Fundus Images

Ekta Gupta,Varun Gupta,Muskaan Chopra,Prakash Chandra Chhipa,Marcus Liwicki

2023-04-20

Abstract:This work presents a novel label-efficient selfsupervised representation learning-based approach for classifying diabetic retinopathy (DR) images in cross-domain settings. Most of the existing DR image classification methods are based on supervised learning which requires a lot of time-consuming and expensive medical domain experts-annotated data for training. The proposed approach uses the prior learning from the source DR image dataset to classify images drawn from the target datasets. The image representations learned from the unlabeled source domain dataset through contrastive learning are used to classify DR images from the target domain dataset. Moreover, the proposed approach requires a few labeled images to perform successfully on DR image classification tasks in cross-domain settings. The proposed work experiments with four publicly available datasets: EyePACS, APTOS 2019, MESSIDOR-I, and Fundus Images for self-supervised representation learning-based DR image classification in cross-domain settings. The proposed method achieves state-of-the-art results on binary and multiclassification of DR images, even in cross-domain settings. The proposed method outperforms the existing DR image binary and multi-class classification methods proposed in the literature. The proposed method is also validated qualitatively using class activation maps, revealing that the method can learn explainable image representations. The source code and trained models are published on GitHub.

Image and Video Processing,Computer Vision and Pattern Recognition
Category-aware Feature Attribution for Self-Optimizing Medical Image Classification

Jie Lei,Guoyu Yang,Shuaiwei Wang,Zunlei Feng,Ronghua Liang

DOI: https://doi.org/10.1016/j.displa.2023.102397

IF: 3.074

2023-01-01

Displays

Abstract:Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image classification. Although remarkable advancement has been made, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the unavoidable hard samples and minority class samples. Meanwhile, it is hard to understand the prediction mechanism and debug the wrong prediction. To address these issues, this paper proposes a Self-Optimizing medical image classification framework based on category-aware feature attribution, which automatically performs model diagnosis, repairing, and enhancement. On one hand, we introduce selective filter restraining to alleviate the disturbance of misclassified features for hard samples. On the other hand, we introduce sample-specific fine-tuning to promote category-wise feature learning, especially for minority classes. The Self-Optimizing framework is a versatile module for other medical image classifiers based on mainstream CNN structures. We extensively validated the proposed method on three typical yet challenging medical image classification tasks: (i) skin lesion classification from dermoscopic images in the International Skin Imaging Collaboration (ISIC) 2019 dataset, (ii) breast cancer classification from ultrasound images in Breast Ultrasound Images (BUSI) dataset, and (iii) breast tumour classification from histopathologic scans of lymph node sections in PatchCamelyon (PCam) dataset. Experimental results demonstrate that the proposed framework achieves 1%∼5% performance gain for various classifiers, brings better explainability, and can be applied to incremental learning scenarios.
Self-supervised learning for medical image analysis: Discriminative, restorative, or adversarial?

Fatemeh Haghighi,Mohammad Reza Hosseinzadeh Taher,Michael B. Gotway,Jianming Liang

DOI: https://doi.org/10.1016/j.media.2024.103086

IF: 10.9

2024-01-30

Medical Image Analysis

Abstract:Discriminative, restorative, and adversarial learning have proven beneficial for self-supervised learning schemes in computer vision and medical imaging. Existing efforts, however, fail to capitalize on the potentially synergistic effects these methods may offer in a ternary setup, which, we envision can significantly benefit deep semantic representation learning. Towards this end, we developed DiRA, the first framework that unites discriminative, restorative, and adversarial learning in a unified manner to collaboratively glean complementary visual information from unlabeled medical images for fine-grained semantic representation learning. Our extensive experiments demonstrate that DiRA: (1) encourages collaborative learning among three learning ingredients, resulting in more generalizable representation across organs, diseases, and modalities; (2) outperforms fully supervised ImageNet models and increases robustness in small data regimes, reducing annotation cost across multiple medical imaging applications; (3) learns fine-grained semantic representation, facilitating accurate lesion localization with only image-level annotation; (4) improves reusability of low/mid-level features; and (5) enhances restorative self-supervised approaches, revealing that DiRA is a general framework for united representation learning. Code and pretrained models are available at https://github.com/JLiangLab/DiRA .

engineering, biomedical,computer science, interdisciplinary applications, artificial intelligence,radiology, nuclear medicine & medical imaging
Effect of Rotation Angle in Self-Supervised Pre-training is Dataset-Dependent

Amy Saranchuk,Michael Guerzhoy

2024-06-21

Abstract:Self-supervised learning for pre-training (SSP) can help the network learn better low-level features, especially when the size of the training set is small. In contrastive pre-training, the network is pre-trained to distinguish between different versions of the input. For example, the network learns to distinguish pairs (original, rotated) of images where the rotated image was rotated by angle $\theta$ vs. other pairs of images. In this work, we show that, when training using contrastive pre-training in this way, the angle $\theta$ and the dataset interact in interesting ways. We hypothesize, and give some evidence, that, for some datasets, the network can take "shortcuts" for particular rotation angles $\theta$ based on the distribution of the gradient directions in the input, possibly avoiding learning features other than edges, but our experiments do not seem to support that hypothesis. We demonstrate experiments on three radiology datasets. We compute the saliency map indicating which pixels were important in the SSP process, and compare the saliency map to the ground truth foreground/background segmentation. Our visualizations indicate that the effects of rotation angles in SSP are dataset-dependent. We believe the distribution of gradient orientations may play a role in this, but our experiments so far are inconclusive.

Computer Vision and Pattern Recognition
Rotation and Translation Invariant Representation Learning with Implicit Neural Representations

Sehyun Kwon,Joo Young Choi,Ernest K. Ryu

2023-06-12

Abstract:In many computer vision applications, images are acquired with arbitrary or random rotations and translations, and in such setups, it is desirable to obtain semantic representations disentangled from the image orientation. Examples of such applications include semiconductor wafer defect inspection, plankton microscope images, and inference on single-particle cryo-electron microscopy (cryo-EM) micro-graphs. In this work, we propose Invariant Representation Learning with Implicit Neural Representation (IRL-INR), which uses an implicit neural representation (INR) with a hypernetwork to obtain semantic representations disentangled from the orientation of the image. We show that IRL-INR can effectively learn disentangled semantic representations on more complex images compared to those considered in prior works and show that these semantic representations synergize well with SCAN to produce state-of-the-art unsupervised clustering results.

Computer Vision and Pattern Recognition,Artificial Intelligence
Achieving Rotation Invariance in Convolution Operations: Shifting from Data-Driven to Mechanism-Assured

Hanlin Mo,Guoying Zhao

2024-04-17

Abstract:Achieving rotation invariance in deep neural networks without relying on data has always been a hot research topic. Intrinsic rotation invariance can enhance the model's feature representation capability, enabling better performance in tasks such as multi-orientation object recognition and detection. Based on various types of non-learnable operators, including gradient, sort, local binary pattern, maximum, etc., this paper designs a set of new convolution operations that are natually invariant to arbitrary rotations. Unlike most previous studies, these rotation-invariant convolutions (RIConvs) have the same number of learnable parameters and a similar computational process as conventional convolution operations, allowing them to be interchangeable. Using the MNIST-Rot dataset, we first verify the invariance of these RIConvs under various rotation angles and compare their performance with previous rotation-invariant convolutional neural networks (RI-CNNs). Two types of RIConvs based on gradient operators achieve state-of-the-art results. Subsequently, we combine RIConvs with different types and depths of classic CNN backbones. Using the OuTex_00012, MTARSI, and NWPU-RESISC-45 datasets, we test their performance on texture recognition, aircraft type recognition, and remote sensing image classification tasks. The results show that RIConvs significantly improve the accuracy of these CNN backbones, especially when the training data is limited. Furthermore, we find that even with data augmentation, RIConvs can further enhance model performance.

Computer Vision and Pattern Recognition
Self-Supervised Learning Featuring Small-Scale Image Dataset for Treatable Retinal Diseases Classification

Luffina C. Huang,Darren J. Chiu,Manish Mehta

2024-04-16

Abstract:Automated medical diagnosis through image-based neural networks has increased in popularity and matured over years. Nevertheless, it is confined by the scarcity of medical images and the expensive labor annotation costs. Self-Supervised Learning (SSL) is an good alternative to Transfer Learning (TL) and is suitable for imbalanced image datasets. In this study, we assess four pretrained SSL models and two TL models in treatable retinal diseases classification using small-scale Optical Coherence Tomography (OCT) images ranging from 125 to 4000 with balanced or imbalanced distribution for training. The proposed SSL model achieves the state-of-art accuracy of 98.84% using only 4,000 training images. Our results suggest the SSL models provide superior performance under both the balanced and imbalanced training scenarios. The SSL model with MoCo-v2 scheme has consistent good performance under the imbalanced scenario and, especially, surpasses the other models when the training set is less than 500 images.

Computer Vision and Pattern Recognition,Machine Learning
Local-to-Global Self-Supervised Representation Learning for Diabetic Retinopathy Grading

Mostafa Hajighasemloua,Samad Sheikhaei,Hamid Soltanian-Zadeha

2024-10-01

Abstract:Artificial intelligence algorithms have demonstrated their image classification and segmentation ability in the past decade. However, artificial intelligence algorithms perform less for actual clinical data than those used for simulations. This research aims to present a novel hybrid learning model using self-supervised learning and knowledge distillation, which can achieve sufficient generalization and robustness. The self-attention mechanism and tokens employed in ViT, besides the local-to-global learning approach used in the hybrid model, enable the proposed algorithm to extract a high-dimensional and high-quality feature space from images. To demonstrate the proposed neural network's capability in classifying and extracting feature spaces from medical images, we use it on a dataset of Diabetic Retinopathy images, specifically the EyePACS dataset. This dataset is more complex structurally and challenging regarding damaged areas than other medical images. For the first time in this study, self-supervised learning and knowledge distillation are used to classify this dataset. In our algorithm, for the first time among all self-supervised learning and knowledge distillation models, the test dataset is 50% larger than the training dataset. Unlike many studies, we have not removed any images from the dataset. Finally, our algorithm achieved an accuracy of 79.1% in the linear classifier and 74.36% in the k-NN algorithm for multiclass classification. Compared to a similar state-of-the-art model, our results achieved higher accuracy and more effective representation spaces.

Computer Vision and Pattern Recognition,Image and Video Processing
OCT-SelfNet: A Self-Supervised Framework with Multi-Modal Datasets for Generalized and Robust Retinal Disease Detection

Fatema-E Jannat,Sina Gholami,Minhaj Nur Alam,Hamed Tabkhi

2024-01-23

Abstract:Despite the revolutionary impact of AI and the development of locally trained algorithms, achieving widespread generalized learning from multi-modal data in medical AI remains a significant challenge. This gap hinders the practical deployment of scalable medical AI solutions. Addressing this challenge, our research contributes a self-supervised robust machine learning framework, OCT-SelfNet, for detecting eye diseases using optical coherence tomography (OCT) images. In this work, various data sets from various institutions are combined enabling a more comprehensive range of representation. Our method addresses the issue using a two-phase training approach that combines self-supervised pretraining and supervised fine-tuning with a mask autoencoder based on the SwinV2 backbone by providing a solution for real-world clinical deployment. Extensive experiments on three datasets with different encoder backbones, low data settings, unseen data settings, and the effect of augmentation show that our method outperforms the baseline model, Resnet-50 by consistently attaining AUC-ROC performance surpassing 77% across all tests, whereas the baseline model exceeds 54%. Moreover, in terms of the AUC-PR metric, our proposed method exceeded 42%, showcasing a substantial increase of at least 10% in performance compared to the baseline, which exceeded only 33%. This contributes to our understanding of our approach's potential and emphasizes its usefulness in clinical settings.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
DiRA: Discriminative, Restorative, and Adversarial Learning for Self-supervised Medical Image Analysis

Fatemeh Haghighi,Mohammad Reza Hosseinzadeh Taher,Michael B. Gotway,Jianming Liang

DOI: https://doi.org/10.48550/arXiv.2204.10437

2022-04-22

Abstract:Discriminative learning, restorative learning, and adversarial learning have proven beneficial for self-supervised learning schemes in computer vision and medical imaging. Existing efforts, however, omit their synergistic effects on each other in a ternary setup, which, we envision, can significantly benefit deep semantic representation learning. To realize this vision, we have developed DiRA, the first framework that unites discriminative, restorative, and adversarial learning in a unified manner to collaboratively glean complementary visual information from unlabeled medical images for fine-grained semantic representation learning. Our extensive experiments demonstrate that DiRA (1) encourages collaborative learning among three learning ingredients, resulting in more generalizable representation across organs, diseases, and modalities; (2) outperforms fully supervised ImageNet models and increases robustness in small data regimes, reducing annotation cost across multiple medical imaging applications; (3) learns fine-grained semantic representation, facilitating accurate lesion localization with only image-level annotation; and (4) enhances state-of-the-art restorative approaches, revealing that DiRA is a general mechanism for united representation learning. All code and pre-trained models are available at https: //github.com/JLiangLab/DiRA.

Computer Vision and Pattern Recognition,Image and Video Processing
Big Self-Supervised Models Advance Medical Image Classification

Shekoofeh Azizi,Basil Mustafa,Fiona Ryan,Zachary Beaver,Jan Freyberg,Jonathan Deaton,Aaron Loh,Alan Karthikesalingam,Simon Kornblith,Ting Chen,Vivek Natarajan,Mohammad Norouzi

DOI: https://doi.org/10.1109/iccv48922.2021.00346

2021-10-01

Abstract:Self-supervised pretraining followed by supervised fine-tuning has seen success in image recognition, especially when labeled examples are scarce, but has received limited attention in medical image analysis. This paper studies the effectiveness of self-supervised learning as a pre-training strategy for medical image classification. We conduct experiments on two distinct tasks: dermatology condition classification from digital camera images and multi-label chest X-ray classification, and demonstrate that self-supervised learning on ImageNet, followed by additional self-supervised learning on unlabeled domain-specific medical images significantly improves the accuracy of medical image classifiers. We introduce a novel Multi-Instance Contrastive Learning (MICLe) method that uses multiple images of the underlying pathology per patient case, when available, to construct more informative positive pairs for self-supervised learning. Combining our contributions, we achieve an improvement of 6.7% in top-1 accuracy and an improvement of 1.1% in mean AUC on dermatology and chest X-ray classification respectively, outperforming strong supervised baselines pretrained on ImageNet. In addition, we show that big self-supervised models are robust to distribution shift and can learn efficiently with a small number of labeled medical images.

Explorer Atypical Femoral Fracture in Osteoporosis Pseudoglioma Syndrome Associated With Two Novel Compound Heterozygous Mutations in LRP 5

IRLSG: Invariant Representation Learning for Single-Domain Generalization in Medical Image Segmentation

Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis

Examining the quality of learned representations in self-supervised medical image analysis: a comprehensive review and empirical study

OPTiML: Dense Semantic Invariance Using Optimal Transport for Self-Supervised Medical Image Representation

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging

A rotation meanout network with invariance for dermoscopy image classification and retrieval

Unsupervised Representation Learning by Predicting Image Rotations

Self-Supervised Representation Learning by Rotation Feature Decoupling

3D Self-Supervised Methods for Medical Imaging

Learning Self-Supervised Representations for Label Efficient Cross-Domain Knowledge Transfer on Diabetic Retinopathy Fundus Images

Category-aware Feature Attribution for Self-Optimizing Medical Image Classification

Self-supervised learning for medical image analysis: Discriminative, restorative, or adversarial?

Effect of Rotation Angle in Self-Supervised Pre-training is Dataset-Dependent

Rotation and Translation Invariant Representation Learning with Implicit Neural Representations

Achieving Rotation Invariance in Convolution Operations: Shifting from Data-Driven to Mechanism-Assured

Self-Supervised Learning Featuring Small-Scale Image Dataset for Treatable Retinal Diseases Classification

Local-to-Global Self-Supervised Representation Learning for Diabetic Retinopathy Grading

OCT-SelfNet: A Self-Supervised Framework with Multi-Modal Datasets for Generalized and Robust Retinal Disease Detection

DiRA: Discriminative, Restorative, and Adversarial Learning for Self-supervised Medical Image Analysis

Big Self-Supervised Models Advance Medical Image Classification