Abstract:Whole slide image (WSI) classification is a critical task in computational pathology, requiring the processing of gigapixel-sized images, which is challenging for current deep-learning methods. Current state of the art methods are based on multi-instance learning schemes (MIL), which usually rely on pretrained features to represent the instances. Due to the lack of task-specific annotated data, these features are either obtained from well-established backbones on natural images, or, more recently from self-supervised models pretrained on histopathology. However, both approaches yield task-agnostic features, resulting in performance loss compared to the appropriate task-related supervision, if available. In this paper, we show that when task-specific annotations are limited, we can inject such supervision into downstream task training, to reduce the gap between fully task-tuned and task agnostic features. We propose Prompt-MIL, an MIL framework that integrates prompts into WSI classification. Prompt-MIL adopts a prompt tuning mechanism, where only a small fraction of parameters calibrates the pretrained features to encode task-specific information, rather than the conventional full fine-tuning approaches. Extensive experiments on three WSI datasets, TCGA-BRCA, TCGA-CRC, and BRIGHT, demonstrate the superiority of Prompt-MIL over conventional MIL methods, achieving a relative improvement of 1.49%-4.03% in accuracy and 0.25%-8.97% in AUROC while using fewer than 0.3% additional parameters. Compared to conventional full fine-tuning approaches, we fine-tune less than 1.3% of the parameters, yet achieve a relative improvement of 1.29%-13.61% in accuracy and 3.22%-27.18% in AUROC and reduce GPU memory consumption by 38%-45% while training 21%-27% faster. Our code is available at <a class="link-external link-https" href="https://github.com/cvlab-stonybrook/PromptMIL" rel="external noopener nofollow">this https URL</a>.

Protein Multimer Structure Prediction via Prompt Learning

Multi-level Protein Structure Pre-training via Prompt Learning

DeepTrio: a Ternary Prediction System for Protein-Protein Interaction Using Mask Multiple Parallel Convolutional Neural Networks

Estimating protein complex model accuracy based on ultrafast shape recognition and deep learning in CASP15.

Linker-Tuning: Optimizing Continuous Prompts for Heterodimeric Protein Prediction

Prompt-Guided Injection of Conformation to Pre-trained Protein Model

Enhancing AlphaFold-Multimer-based Protein Complex Structure Prediction with MULTICOM in CASP15

MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training

Integrating deep learning, threading alignments, and a multi‐MSA strategy for high‐quality protein monomer and complex structure prediction in CASP15

So ManyFolds, So Little Time: Efficient Protein Structure Prediction With pLMs and MSAs

Prompt-MIL: Boosting Multi-Instance Learning Schemes via Task-specific Prompt Tuning

PSC-CPI: Multi-Scale Protein Sequence-Structure Contrasting for Efficient and Generalizable Compound-Protein Interaction Prediction

MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Prediction

PSSP-MFFNet: A Multifeature Fusion Network for Protein Secondary Structure Prediction

DPLM-2: A Multimodal Diffusion Protein Language Model

Deep Reinforcement Learning for Modelling Protein Complexes

Learning to Predict Mutation Effects of Protein-Protein Interactions by Microenvironment-aware Hierarchical Prompt Learning

HelixFold-Multimer: Elevating Protein Complex Structure Prediction to New Heights

Multimodal Pre-Training Model for Sequence-based Prediction of Protein-Protein Interaction

MMCL-CPI: A multi-modal compound-protein interaction prediction model incorporating contrastive learning pre-training

Multimodal pretraining for unsupervised protein representation learning