Abstract:Abstract Summary The biological functions of proteins are determined by the chemical and geometric properties of their surfaces. Recently, with the booming progress of deep learning, a series of learning-based surface descriptors have been proposed and achieved inspirational performance in many tasks such as protein design, protein-protein interaction prediction, etc. However, they are still limited by the problem of label scarcity, since the labels are typically obtained through wet experiments. Inspired by the great success of self-supervised learning in natural language processing and computer vision, we introduce ProteinMAE, a self-supervised framework specifically designed for protein surface representation to mitigate label scarcity. Specifically, we propose an efficient network and utilize a large number of accessible unlabeled protein data to pre-train it by self-supervised learning. Then we use the pre-trained weights as initialization and fine-tune the network on downstream tasks. To demonstrate the effectiveness of our method, we conduct experiments on three different downstream tasks including binding site identification in protein surface, ligand binding protein pocket classification, and protein-protein interaction prediction. The extensive experiments show that our method not only successfully improves the network’s performance on all downstream tasks and enables it to achieve competitive performance with state-of-the-art methods. Moreover, our proposed network also exhibits significant advantages in terms of computational cost, which only requires less than a tenth of memory cost of previous methods. Availability and implementation https://github.com/phdymz/ProteinMAE. Supplementary information We provide supplementary material following the manuscript.

SS-Pro: a simplified Siamese contrastive learning approach for protein surface representation

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

Siamese Prototypical Contrastive Learning

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Learning Complete Protein Representation by Deep Coupling of Sequence and Structure

ProteinMAE: Masked Autoencoder for Protein Surface Self-supervised Learning

CCPL: Cross-modal Contrastive Protein Learning

AtomSurf : Surface Representation for Learning on Protein Structures

Contrastive Representation Learning for 3D Protein Structures

SiameseCPP: a sequence-based Siamese network to predict cell-penetrating peptides by contrastive learning

Crafting Better Contrastive Views for Siamese Representation Learning

Exploring Simple Siamese Representation Learning

Siamese Image Modeling for Self-Supervised Vision Representation Learning

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

EnSiam: Self-Supervised Learning With Ensemble Representations

Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction

A Simple Framework for Contrastive Learning of Visual Representations

Contrastive Learning of Graph-Kernel-based Representations for Proteins.

Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning

CSNet: Contrastive Siamese Network for Robust SLU.

SiSL-Net: Saliency-guided self-supervised learning network for image classification