FAST: A Dual-tier Few-Shot Learning Paradigm for Whole Slide Image Classification

Kexue Fu,Xiaoyuan Luo,Linhao Qu,Shuo Wang,Ying Xiong,Ilias Maglogiannis,Longxiang Gao,Manning Wang
2024-09-29
Abstract:The expensive fine-grained annotation and data scarcity have become the primary obstacles for the widespread adoption of deep learning-based Whole Slide Images (WSI) classification algorithms in clinical practice. Unlike few-shot learning methods in natural images that can leverage the labels of each image, existing few-shot WSI classification methods only utilize a small number of fine-grained labels or weakly supervised slide labels for training in order to avoid expensive fine-grained annotation. They lack sufficient mining of available WSIs, severely limiting WSI classification performance. To address the above issues, we propose a novel and efficient dual-tier few-shot learning paradigm for WSI classification, named FAST. FAST consists of a dual-level annotation strategy and a dual-branch classification framework. Firstly, to avoid expensive fine-grained annotation, we collect a very small number of WSIs at the slide level, and annotate an extremely small number of patches. Then, to fully mining the available WSIs, we use all the patches and available patch labels to build a cache branch, which utilizes the labeled patches to learn the labels of unlabeled patches and through knowledge retrieval for patch classification. In addition to the cache branch, we also construct a prior branch that includes learnable prompt vectors, using the text encoder of visual-language models for patch classification. Finally, we integrate the results from both branches to achieve WSI classification. Extensive experiments on binary and multi-class datasets demonstrate that our proposed method significantly surpasses existing few-shot classification methods and approaches the accuracy of fully supervised methods with only 0.22$\%$ annotation costs. All codes and models will be publicly available on <a class="link-external link-https" href="https://github.com/fukexue/FAST" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in clinical practice, due to the high cost of fine - grained annotation and data scarcity, existing deep - learning methods are difficult to be widely applied to whole slide image (WSI) classification. Specifically: 1. **The cost problem of fine - grained annotation**: Fine - grained annotation of WSI requires expert knowledge and is very expensive. 2. **Data scarcity**: Only a limited number of WSIs can be obtained in many clinical scenarios, which limits the supervised information available for model training. To solve these problems, the authors propose a new two - level few - shot learning paradigm named FAST (Few - shot learning paradigm for Whole Slide Image Classification), aiming to achieve high - precision WSI classification with a lower annotation cost and quickly adapt to various WSI classification tasks. Specifically, FAST solves the problems in the following ways: - **Two - level annotation strategy**: Select a small number of WSIs and annotate a very small number of patches in each selected WSI, thereby significantly reducing the cost of fine - grained annotation. - **Two - branch classification framework**: - **Cache branch**: Use all patches and their available labels to build a cache model, and classify unannotated patches through knowledge retrieval. - **Prior branch**: Utilize the text encoder of a vision - language model (such as CLIP) to generate category - specific prompts, and design a learnable vision - language classifier through prompt - learning techniques. In this way, FAST can achieve classification performance close to that of fully - supervised methods with only 0.22% of the annotation cost. This makes FAST very suitable for application in cases of data scarcity and high annotation cost. ### Summary The main contribution of this paper lies in proposing a new few - shot learning paradigm FAST. Through an efficient two - level annotation strategy and a parameter - efficient two - branch classification framework, it solves the problems of high cost of fine - grained annotation and data scarcity in WSI classification, and achieves high - precision WSI classification.