Foundation model for cancer imaging biomarkers

Suraj Pai,Dennis Bontempi,Ibrahim Hadzic,Vasco Prudente,Mateo Sokač,Tafadzwa L. Chaunzwa,Simon Bernatz,Ahmed Hosny,Raymond H. Mak,Nicolai J. Birkbak,Hugo J. W. L. Aerts

DOI: https://doi.org/10.1038/s42256-024-00807-9

IF: 23.8

2024-03-16

Nature Machine Intelligence

Abstract:Foundation models in deep learning are characterized by a single large-scale model trained on vast amounts of data serving as the foundation for various downstream tasks. Foundation models are generally trained using self-supervised learning and excel in reducing the demand for training samples in downstream applications. This is especially important in medicine, where large labelled datasets are often scarce. Here, we developed a foundation model for cancer imaging biomarker discovery by training a convolutional encoder through self-supervised learning using a comprehensive dataset of 11,467 radiographic lesions. The foundation model was evaluated in distinct and clinically relevant applications of cancer imaging-based biomarkers. We found that it facilitated better and more efficient learning of imaging biomarkers and yielded task-specific models that significantly outperformed conventional supervised and other state-of-the-art pretrained implementations on downstream tasks, especially when training dataset sizes were very limited. Furthermore, the foundation model was more stable to input variations and showed strong associations with underlying biology. Our results demonstrate the tremendous potential of foundation models in discovering new imaging biomarkers that may extend to other clinical use cases and can accelerate the widespread translation of imaging biomarkers into clinical settings.

computer science, artificial intelligence, interdisciplinary applications

What problem does this paper attempt to address?

This paper explores the application of foundation models in the discovery of cancer imaging biomarkers. Foundation models are large-scale deep learning models that are pretrained on a large amount of unlabeled data and can serve as the basis for various downstream tasks. In the medical field, especially in medical image analysis, obtaining a large amount of annotated data is often challenging, and foundation models can reduce the need for labeled training samples.

The researchers trained a convolutional encoder as the foundation model, pretraining it through self-supervised learning on a comprehensive dataset consisting of 11,467 radiological lesions. They then evaluated this model in three different clinically relevant applications: 1) lesion anatomical site classification, 2) lung nodule malignancy classification, and 3) overall survival classification of non-small cell lung cancer (NSCLC). The study found that the foundation model performed well in these tasks, especially in cases with limited data, outperforming traditional supervised learning methods and other state-of-the-art pretrained implementations.

The paper also compared different pretraining strategies, such as autoencoders and SimCLR, and the results showed that a modified version of SimCLR pretraining strategy performed best in terms of balancing accuracy and mean precision. Furthermore, the foundation model maintained stable performance even with reduced data, which is particularly important in resource-limited settings.

In the task of lung cancer prognosis prediction, the foundation model also demonstrated advantages, with its feature extraction followed by a linear classifier outperforming other baseline methods. The study also analyzed the input stability and test-retest consistency of the model, and validated the relevance of the model to tumor biology through gene expression data correlation analysis.

Overall, this paper demonstrates the tremendous potential of foundation models in discovering new imaging biomarkers, especially in situations with small datasets, which may accelerate the widespread application of imaging biomarkers in clinical practice.